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Abstract 

It is well known that the linear stability of Lagrangian elliptic equilateral triangle homographic solu- 
tions in the classical planar three-body problem depends on the mass parameter /3 = 21{m\ni2 + OT2W3 + 
OT3OTi)/(oti +m.2+m-i) 2 6 [0, 9] and the eccentricity e e [0, 1). We are not aware of any existing analytical 
method which relates the linear stability of these solutions to these two parameters directly in the full 
rectangle [0,9] x [0, 1), besides perturbation methods for e > small enough, blow-up techniques for 
e sufficiently close to 1, and numerical studies. In this paper, we introduce a new rigorous analytical 
method to study the linear stability of these solutions in terms of the two parameters in the full (J3, e) 
range [0, 9] x [0, 1) via the w-index theory of symplectic paths for to belonging to the unit circle of the 
complex plane, and the theory of linear operators. After establishing the w-index decreasing property 
of the solutions in /3 for fixed e e [0, 1), we prove the existence of three curves located from left to 
right in the rectangle [0,9] x [0, 1), among which two are -1 degeneracy curves and the third one is 
the right envelop curve of the w-degeneracy curves for u> + 1, and show that the linear stability pattern 
of such elliptic Lagrangian solutions changes if and only if the parameter (J3, e) passes through each of 
these three curves. Interesting symmetries of these curves are also observed. The singular case when the 
eccentricity e approaches to 1 is also analyzed in details concerning the linear stability. 

Keywords: planar three-body problem, Lagrangian solution, linear stability, Maslov-type w-index, per- 
turbations of linear operators. 
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1 Introduction and main results 

We consider the classical planar three-body problem in celestial mechanics. Denote by q\,qi,q^, e R 2 the 
position vectors of three particles with masses m\,m2, > respectively. By Newton's second law and the 
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law of universal gravitation, the system of equations for this problem is 

m{qi = ^-, for i- 1,2,3, (1.1) 
oqi 

where U{q) = U{q\,q2,q3) = Zi<i</<3 \\q'-q\\ * s tne potential or force function by using the standard norm 
|| • || of vector in R 2 . For periodic solutions with period 2n, the system is the Euler-Lagrange equation of the 
action functional 
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defined on the loop space W l ' 2 (KI2nX,X), where 
:=J? = (gi,«i,?3)e(R 2 ) 3 
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+ U(q(t)) 



dt 



^ rmqi = 0, qi * qj, Vi * j 



is the configuration space of the planar three-body problem. The periodic solutions of (11.11 ) correspond to 
critical points of the action functional. 

It is a well-known fact that (11.1b can be reformulated as a Hamiltonian system. Let pi,p2,P3 £ R 2 be 
the momentum vectors of the particles respectively. The Hamiltonian system corresponding to (11.11 is 

dH dH 

Pi - —7 — > qi = ^ — , for i = 1,2,3, (1.2) 
oqi dpi 

with Hamiltonian function 

H(p,q) = H(j>i,p 2 ,p3,qi,q2,q3) = ^ ~ <?2, <?3)- (1.3) 
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In 1772, Lagrange ((71) discovered some celebrated periodic solutions, now named after him, to the 
planar three-body problem, namely the three bodies form an equilateral triangle at any instant of the motion 
and at the same time each body travels along a specific Keplerian elliptic orbit about the center of masses of 
the system. All these orbits are homographic solutions. 

When < e < 1, the Keplerian orbit is elliptic, following Meyer and Schmidt ( lfl"5l ). we call such 
elliptic Lagrangian solutions elliptic relative equilibria. Specially when e = 0, the Keplerian elliptic motion 
becomes circular motion and then all the three bodies move around the center of masses along circular orbits 
with the same frequency, which are called relative equilibria traditionally. 

Our main concern in this paper is the linear stability of these homographic solutions. For the planar 
three-body problem with masses m\,ni2, > 0, it turns out that the stability of elliptic Lagrangian solutions 
depends on two parameters, namely the mass parameter f5 € [0, 9] defined below and the eccentricity e e 
[0, 1), 

21{m\m2 + m\tn-i + 1112111^) 
(mi + ni2 + m^) 2 

Note that besides local perturbation method or blow up technique which study only the case for small enough 
e > or e < 1 sufficiently close to 1 , we are not aware of any rigorous analytical method dealing with this 
problem for the major part of the full range of the (/?, e) rectangle [0, 9] x [0, 1), except the recent paper [[5) of 
the first and the third named authors. Continuing with [4] and |5 ], the current paper is devoted to introduce 
a new rigorous analytical method to study the linear stability of the elliptic Lagrangian solutions in the full 
range of the (J3, e) rectangle [0, 9] x [0, 1) via the index theory of symplectic paths and the perturbation theoiy 
of linear operators. 
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Figure 1: Stability bifurcation diagram of Lagrangian equilateral triangular homographic orbits in the (fi, e) 
rectangle [0, 9] x [0, 1). It is the Fig. 5 in [[3). Here the regions I, II, III, IV, V and VI are EE, EE, EH, HH, 
HH and CS respectively. 



The linear stability of relative equilibria were known more than a century ago and it is due to Gascheau 
(0, 1843) and Routh ( ifTTl , 1875) independently. In this case, using the Floquet theory one can work out 
all the details explicitly by hands. 

After initial considerations of Danby (Q], 1964), Roberts ( fT6l , 2002) reduced all the symmetries of the 
problem and their first integrals and studied the case of sufficiently small e > by perturbation techniques. 
He also got a partial stability bifurcation diagram for this case, where the stability patterns are clearly 
presented. 

In 2005, Meyer and Schmidt (cf. |fT5l ) used heavily the central configuration nature of the elliptic 
Lagrangian orbits and decomposed the fundamental solution of the elliptic Lagrangian orbit into two parts 
symplectically, one of which is the same as that of the Keplerian solution and the other is the essential part 
for the stability. In the current paper, the fundamental solution of the linearized Hamiltonian system of the 
essential part of the elliptic Lagrangian orbit is denoted by yp, e (t) for t € [0, 2n], which is a path of 4 x 4 
symplectic matrices starting from the identity. They also did the stability analysis by normal form theory for 
small enough e > 0. 

In 2004-2006, Martinez, Sama and Simo (H2,|[l3,|[l4]]) studied the stability problem when e > is 
small enough by using normal form theory, and e < 1 and close to 1 enough by using blow-up technique in 
general homogeneous potential. They further gave a much more complete bifurcation diagram numerically 
and a beautiful figure was drawn there for the full (/?, e) range, which we repeat here as Figure 1 . It is one of 
our primary motivations to understand this diagram globally and analytically in the present work. 

Let U denote the unit circle in the complex plane C. As in |fl3l . the following notations for the different 
parameter regions are used in Figure 1 : 

• EE (elliptic-elliptic), if yp_ e {2n) possesses two pairs of eigenvalues in U \ R; 

• EH (elliptic-hyperbolic), if y/?, e (27r) possesses a pair of eigenvalues in U \ R and a pair of eigenvalues 
inR\{0,±l}; 

• HH (hyperbolic-hyperbolic), if o-(y^ e (2n)) c R\{0,±1}; 

• CS (complex-saddle), if criy^ e {2n)) c C \ (U U R). 

In summary, after these authors, the following results are rigorously proved: 
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(i) the relative equilibrium, i.e., the case of e - 0, is linearly stable if and only if /? < 1; it is only 
spectrally stable and not linearly stable when = 1, and linearly unstable (in fact, CS) when > 1. More 
precisely, when e = and goes from to 1, the two pairs of elliptic characteristics starting from two pairs 
of characteristics +1 move clock-wisely around the unit circle with different speeds, one pair of which will 
arrive at - 1 of the unit circle when = 3/4, and then continuing again in different speeds until both of the 
two pairs of eigenvalues collide on the unit circle when 0=1, then become CS when > 1. When = 9, 
they become a pair of positive double eigenvalues. So when e = the only possible bifurcation points in the 
(0, e) plane are (3/4, 0) and (1,0). 

(ii) It turns out that if = 3/4 is fixed and e increases from slightly, the pair of - 1 characteristics 
switches to a real hyperbolic pair, and two period-doubling bifurcation curves born out from the point (0, e) = 
(3/4,0). When = 1, if e increases from slightly, the two pairs of corresponding elliptic characteristics 
collide and a Krein collision curve bifurcates from (0,e) = (1,0) for such small enough e > 0. An interesting 
phenomenon occurs here, namely, when is slightly larger than 1 , some of the elliptic relative equilibrium 
with e > small can be linearly stable even though the relative equilibrium with e = is not. 

(iii) When e < 1 and e is close to 1 enough, the relative equilibria are all HH under some non-degenerate 
conditions, which is not satisfied at = 6 by numerical computations. 

(iv) But the major part of the intermediate region in the {0, e) rectangle [0, 9] x [0, 1) is totally unknown 
theoretically, besides numerical results. 

Inspired by the second named author's works on the index iteration theory of periodic orbits of Hamilto- 
nian systems (cf. EH), the first and the third named authors initiated the program of applying ideas of index 
theory and its iteration theory in calculus of variations (cf. ifTTl ) to the stability problem of periodic orbits in 
celestial mechanics, specially the elliptical Lagrangian solutions ((H) as well as the celebrated Figure-eight 
periodic orbits due to Chenciner and Montgomery in the planar three-body problem (0]). In 0, the stability 
of elliptical Lagrangian solutions is studied and related to the Morse indies of their iterations, i.e., Theorems 
2.4 and 2.5 below. But in 0, specially the 1 -non-degeneracy of elliptic Lagrangian solutions is not proved, 
and the separation curves of different index regions and thus the stability regions in [0,9] x [0, 1) are not 
identified. 

In the current paper, we develop a new method using w-index theory for symplectic paths introduced by 
C. Conley, E. Zehnder and Y. Long when co = 1 (cf. IfTTl ) and by Y. Long when co e U \ {1} in (9l and 
linear differential operator theory to understand the linear stability diagram of elliptic Lagrangian solutions 
theoretically in the full range of {0, e). Specially the main puipose here is to relate such a linear stability 
directly to the two major parameters of the motion: the mass parameter and the eccentricity e. For each 
fixed e e [0, 1), we prove that the co-index ia)(yp, e ) of the essential part of the elliptic Lagrangian solutions 
is non-decreasing in for all co € U. Then we use this important property to prove all these solutions are 
1 -non-degenerate, find the two -1 degeneracy curves and right envelop curve of all cu-degeneracy curves 
for co e U \ {1}, and determine the linear stability of all sub-regions separated by these three curves. This 
establishes rigorously most parts of the linear stability properties observed numerically in Figure 1, and 
find more interesting properties. Note that the symplectic coordinate decomposition of Meyer and Schmidt 
fits quite well with index theory, and our study will concentrate on the fundamental solution yp, e (t) of the 
linearized Hamiltonian system of the essential part of the elliptic Lagrangian orbit for {0, e) e [0, 9] x [0, 1). 

Denote by Sp(2«) the symplectic group of real 2n x 2n matrices. For any co e U and M e Sp(2«), let 
V(o(M) = dime kerc(M - col%^), and M is called co-degenerate (co-non-degenerate respectively) if v w (M) > 
(v w (M) = respectively). When co = 1 and if there is no confusion, we shall simply omit the subindex 1 
and say just degenerate or non-degenerate. Let e(M) be the total algebraic multiplicity of all eigenvalues of 
M on U. We call M e Sp(2n) spectrally stable if e(M) = 2n, and linearly stable if M is spectrally stable and 
semi-simple. A symplectic matrix M is called strongly linearly stable if there is some e > such that all 
symplectic matrices N satisfying \\M - N\\ < e are linearly stable. And M is hyperbolic, if e(M) = 0. 

The following is the first part of our main results in this paper. 
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Theorem 1.1 In the planar three-body problem with masses m\, m2, and mi, > 0, for the elliptic Lagrangian 
solution q = (qi(t), q2(t), qj,{t)) with eccentricity e and mass parameter ft as given in the essential part 
jp^ e {2n) € Sp(4) of the monodromy matrix of the fundamental solution along this orbit is non-degenerate 
for all (fi,e) € (0,9] x [0, 1); and when [3 = 0, it is degenerate. Note that the Maslov-type index satisfies 
h{yp,e) = Ofor all (fi, e) € [0, 9] x [0, 1). 

In the proof of this theorem, we consider the second order differential operators A(fJ, e) (see (12.251 )) 
corresponding to the variational linear equation to the essential part yp, e (t) of its fundamental solution along 
the orbit. The main ingredient of the proof is the non-decreasing property of o>-index proved in Lemma 
14.41 and Corollary 14. 5 1 below for all a> e U, by which we further prove that the operator A(fj, e) is positive 
definite, and thus 1 -non-degenerate. 

The rest part of this paper, specially Theorems 11.21 and 11.81 below, is devoted to rigorous analytical 
studies on the existences and properties of three distinct curves r s , Y m and Yt locating from left to right in 
the parameter (f3, e) rectangle [0,9) x [0, 1). We prove that the linear stability of the essential part y^^(2n) 
of the monodromy matrix and thus that of the elliptic Lagrange solution change precisely when (fj, e) passes 
through each of these three curves, which yields a complete and rigorous understanding of the linear stability 
of the elliptic Lagrange solutions. Note that here yp e (2n) is always linearly unstable on its hyperbolic 
subregion in the (ft, e) rectangle [0,9] x [0, 1), and our Theorems 11.21 and 1 1 . 8 1 do not distinguish the regions 
IV, V, and VI in Figure 1. 

The main idea in the proofs of Theorems 11.21 and 11.81 is the following: By Theorem 11.11 when (/?, e) 
changes, eigenvalues of yp, e (2n) can leave from the unit circle U only from -1 or some Krein collision 
eigenvalues in U \ {+1}. Thus such -1 and Krein collision eigenvalues should correspond to (f3, e) points 
which form the above mentioned three curves T s , Y m and T^. In order to find those (fi, e) such that -1 € 
o~{yp^{2n)), we prove that the -1 index i-i(yp t e) is non-increasing in/? e [0, 9] for fixed e e [0, 1), and takes 
values 2 at (3 = and at f3 = 9, thus there must exist two -1 index strictly decreasing curves T s and T,„, 
each of which intersects every horizontal line e = constant only once for e € [0, 1), and which then yield 
precisely the two -1 degeneracy curves. Next we prove that the hyperbolic region of yp^(2n) in the (ft, e) 
rectangle [0,9] x [0, 1) is connected, and its boundary curve is continuous and thus well defined, which 
is the third curve for determining the linear stability. Here the part of which is different from the curve 
T m is also the curve of Krein collision eigenvalues of y^ e (2n). We prove also that the two curves T s and T m 
come from two real analytic curves and bifurcate out from (3/4,0), the curve starts from (1,0), and all 
of them goes up and tends to the point (0, 1) as e increases from to 1. These three curves were observed 
numerically in |[T3l as shown in the above Figure 1 . 

In this paper for any M and N e Sp(2n), we write M « N if M = P^NP holds for some P e Sp(2«), 
i.e., ,/V can be obtained from M by a symplectic coordinate change. Recall that as defined in Chapter 1 of 
|[m . the normal form of an M e Sp(2n) is the simplest matrix N e Sp(2n) satisfying N » M (cf. Theorem 
1.7.3 on p. 36 of ifTTI ). Recall also that as introduced in Definition 1.8.9 and Theorem 1.8.10 on p.41 of ifTTTl 
(cf. Definition 12.11 below) . the basic normal form of an M e Sp(2n) is the simplest matrix ,/V e Sp(2rc) such 
that dime kerc(N - a)I) = dime kerc(M - col) for all co e U. It yields the homotopically simplest form of M 
based on the normal form of M for eigenvalues in U. Note that studies at the level of basic normal forms of 
yp^ e (2n) is easier and is already good enough for determining the linear stability. But the results at the level 
of normal forms of yp^ e {2n) are stronger than basic normal forms and involve more demonstrations. Here 
we describe our main results in normal forms in Theorems ll.2l and [L8l below. Note that here the symplectic 
direct sum o is given in (12.11 ) and the normal form matrices D(A), R(9), N\(A, a), N2(co, b) and M%(A, c) used 
in Theorems 11.21 and 1 1 . 8 1 can be found in §2.1 below. 

Theorem 1.2 Using notations in the last theorem, for every e e [0,1), the -1 index /-i(y/j, e ) is non- 
increasing, and strictly decreasing only on two values of p - fi\{e) and [3 = ^(e) € (0,9). Define 
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T ; - = {(fii(e), e)\ee [0, 1)} for i = 1 and 2, 

fi s (e) = mm{/3 l (e),p 2 (e)} and fi m (e) = max{/?i(e),/? 2 (e)} for e e [0, 1), 

and 

T s = {(J3Ae), e)\ee [0, 1)} and T m = {(fi m {e), e) \e e [0, 1)}. 
For every e e [0, 1) we define 

fik{e) = sup{£' e [0, 9] | o-( 7fi , e (2n)) nU#0, V fi e [0,fi']}, (1.5) 

and 

Y k = {(fi k {e), e) € [0, 9] X [0, 1) | e e [0, 1)}. (1.6) 

Then T s , Y m and Tkform three curves which possess the following properties. 

(i) < /3i(e) < 9 for i = 1, 2, and both fi = fi\(e) and fi - f5i(e) are real analytic in e e [0, 1); 

(ii) J3\(0) = y62(0) - 3/4 and lim e ^iy6i(e) - lim e _>i^2(«) = 0. The two curves T\ and T 2 are real 
analytic in e, and bifurcate out from (3/4,0) with tangents - V33/4 and V33/4 respectively, thus they are 
different and their intersection points must be isolated if there exist when e € (0, 1); Consequently, T s and 
T m are different piecewise real analytic curves; 

( Hi) We have 

(2, if 0<J3 <&(<?), 
U{yp,e) = \ 1, if Psie) <P< Pnie), (1.7) 
lo, if f3 m (e)</3<9, 

and T s and Y m are precisely the -1 degeneracy curves of the matrix y^ e {2n) in the (fi, e) rectangle [0, 9] X 
[0, 1); 

(iv) There holds j3 s (e) < p m (e) < f3 k (e) < 9 for all e e [0, 1); 

(v) Every matrix jp^ e {2n) is hyperbolic when e (S/t(e), 9] and e € [0, 1), and there holds 

f3 k (e) = M{{3 e [0, 9] | o-(y^ e (2n)) n U = 0}, V e e [0, 1). (1.8) 

Consequently Yk is the boundary curve of the hyperbolic region of yp^ e {2n) in the (fi, e) rectangle [0, 9] X 
[0, 1); 

(vi) Tk is continuous in e e [0, 1); 

(vii) lim^i/^e) = 0; 

(viii) There exists a point e € (0, 1] such that fl m {e) < f3k(e) holds for all e e [0, e). Therefore the curve 
is different from the curve T m at least when e € [0, e). 

(ix) We have yp^{2n) ~ R(9\)oR{92) for some Q\ and 6 2 € (n,2n), and thus it is strongly linearly stable 
on the segment < J3 < fi s (e); 

(x) We have y^ e (2n) « D(A)oR(B)) for some > A + — 1 and 9 € (n,2n), and it is elliptic-hyperbolic, 
and thus linearly unstable on the segment [3 s {e) < fi < f3 m (e). 

(xi) We have ya e (2jr) « R{9\)oR{9 2 ) for some 9\ € (0, n) and 9 2 € (n, 2n) with 2n — 9 2 < 9\, and thus it 
is strongly linearly stable on the segment f3 m (e) < fi < fit(e). 

Here and below in Theorem \1.8\ we write A = Ap^ e and 9 - 6jg >e for short, all of which depend on the 
parameters fi and e. 

By the Bott-type iteration formulas of Maslov-type indices, we can decompose W 2 ' 2 ([0, 2n], R 2 ) into 
two subspaces E\ and E 2 (see (17.31 ) and (17.41) below) according to the boundary conditions. Then using 
the operator A{fi, e) (see (17.91 )) corresponding to the variational equation, we carry out the computations of 
Morse indices of A(fi, e)\E t with i = 1 and 2 via those of A(0, 0)1^ with i = 1 and 2. 

As a corollary, we have immediately 
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Corollary 1.3 For every e € [0, 1), the Lagrangian orbit is strongly linearly stable if/3 > is small enough. 

Furthermore, we can strengthen the conclusion (v) of Theorem 1 1.21 to 

Proposition 1.4 For the equal mass case, i.e., f> = 9, the matrix y^^(2n) is always hyperbolic and possesses 
a pair of positive double eigenvalues A(e) and /1(e) -1 + 1 for every < e < 1. Consequently, the matrix 
yp^ e (2n) is hyperbolic whenever f3 < 9 is sufficiently close to 9. 

We establish Proposition II .41 in §4.1 by the Maslov-type index theoiy and theory of linear differential 
operators. Then we further have the following 

Theorem 1.5 (i) For every co e U \ {1} and e € [0, 1), the co-index iaj(yp,e) is decreasing for f> e [0,9]. 

(ii) There exist precisely two curves in the (J3,e) rectangle [0,9] x [0,1), on which the Maslov-type 
index iaj(y/3, e ) decreases strictly. These two curves are given by j3 = j3\{e,oS) and {3 = fi2(e,oS) for < 
e < 1 respectively, where both f}\(e,co) and ^j{e,co) are real analytic functions in e € [0, 1) and satisfy 
lim^i/JiO?, co) = lim^i^e, co) = 0. 

This is proved in Theorem 16.3 1 below. Similar to the idea in the proof of Theorem 11.21 we know that 
the a>-Morse indices of A(0, e) and A{9, e) are 2 and respectively. The existence of the two w-index 
strictly decreasing curves follows from the monotonicity of the involved operators. Note that o>index strictly 
decreasing is equivalent to the o»-degeneracy of the operator A(fi, e) by our Proposition 16. 1 I below. With the 
aid of Dunford-Taylor integral, the a>-degeneracy of A(fj, e) is related to the spectral problem of another 
compact operator B(e, co) (see (16.51 ) below), and the real analyticity of the two index degeneracy curves 
follows from the theory of operators. 

Although e < does not have physical meaning, we can extend the fundamental solution to the case 
e 6 (-1,1) mathematically and some interesting property of the two degeneracy curves follows. Namely we 
get 

Theorem 1.6 (i) The identity f$\{e, -1) = fiii-e, -1) holds for all e e (-1,1). For fixed co e U \ {-1} and 
i - 1 and 2, the function fii(e, co) is also even in e e (—1, 1) . 

(ii) For e e (-1, 1), the function f$k(e) is also even in e, i.e., f3k(-e) - fik(e) for all e e (-1, 1). Conse- 
quently Yk can be continuously extended to the region [0,9] X (-1,1) as a curve symmetric to the segment 
[0,9]x{0}. 

Theorem II .61 follows from the fact that A(fj,e) is conjugate to A(f3, -e) by a unitary operator. This is 
proved in Theorems 16.41 and |7 . 2 1 belo w. 

Figure 2 represents the case of to = -1 in Theorem 11.61 Following Theorem 11.21 in Figure 1 , the curve 
separating the regions II and III is T s and the curve separating the regions III and the union of I and IV is 
r m . The curves T\ and the mirror of the curve T2 in Theorem 11.21 together give one of the analytic curves 
in Theorem 11.61 and the other one in the Theorem 1 1.61 is derived from the curves T2 and the mirror of F\ 
in Theorem 1 1.21 as indicated in the Figure 2. So we see that the two seemingly unrelated index degeneracy 
curves in Theorem 1 1.2 1 are in fact coming from one degeneracy curve in Theorem 11.61 Note that the curve 
Tfc separates the regions I, IV and VI in Figure 1. Numerical studies onweU are also given in Figure 4 in 
§10. 

When e = 1, the operator A(fj, 1) is singular and its domain is different from that of A(fJ,e) with e < 1, 
and not convenient to be used. Thus we use the corresponding sesquilinear forms to study the limiting case 
when e — > 1 . 

Theorem 1.7 For any fixed < yS < 9, the matrix yp^ e (2n) is hyperbolic when 1 - \e\ is small enough. 
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Figure 2: Stability bifurcation diagram of Lagrangian triangular homographic orbits in the (f3, e) rectangle 
[0, 9] x (-1, 1). It is symmetric with respect ro the /3-axis. The region IV is hyperbolic (HH or CS). 



Note that by definition, at least one pair of eigenvalues of the matrix yp ie {2n) locates on the unit circle U 
when p e [0,fi k (e)] and e e [0, 1). 

For ifi, e) located on these three special curves, we have the following 

Theorem 1.8 For the normal forms ofyp ie (2n) when (J3, e) e Y s , Y m or we have the following results. 

(i) If/3 s (e) < f3 m {e), we have y J g j ( e ) >e (2^) « iVi(-l, l)oR{9)for some 9 e (n, 2n), and thus it is spectrally 
stable and linearly unstable; 

(ii) If f3 s (e) = J3 m (e) < fiife), we have yp s ( e ),e(2x) ~ -I 2 oR(G)for some 6 e (n, 2n), and thus it is linearly 
stable, but not strongly linearly stable; 

(Hi) If [3 s {e) < fi m {e) < fik(e), we have yp m ( e ),eQri) ~ A^i (— 1, - l)oR{9) for some 9 e (n, 2n), and thus it is 
spectrally stable and linearly unstable; 

(iv) Ifp s {e) < p m (e) < k (e), we have y Pk{e) A^) « N 2 (e b) for some 9 e (0, n) and b = ( ^ ^ ) 

satisfying (b 2 - b^) sin 9 > 0, i.e., N 2 (e^~^ e , b) is trivial in the sense of Definition 1.8.11 on p.41 of Mill (cf. 
§2.1 below). Consequently the matrix yp k ( e ), e (2n) is spectrally stable and linearly unstable; 

(v) If J3 s (e) < J3 m (e) = fik(e), we have either yp k ( e ^ e (2n) « N\(-l, 1 )oD(A) for some -1 + A < and 
is linearly unstable; or yp k ( e ), e {2n) « M 2 {—\,c) with c\,c 2 e R and c 2 + 0, and it is spectrally stable and 
linearly unstable; 

(vi) If/3 s (e) = J3 m (e) = Pk{e), either y^ k ^ e (2n) « M 2 (-l, c) with c\, c% e R and c 2 = which possesses 
basic normal form N\{-\, l)oiVi(-l, 1), or yp k ( e \eQn) *iVi(-l, l)oiVi(-l, 1). Thus yp k ( e ) te (2n) is spectrally 
stable and linearly unstable. 

Theorem 1.9 For any fixed e € [0, 1), the set 

I e = {ft € (0, 9] | the spectrum of yp^ e (2n) is CS} 

is a non-empty open set. 

Remark 1.10 Note that our above results yield that the two curves Y s and Y m can have only isolated inter- 
section points, but it is not clear if they do have any when e e (0, 1). It is not clear so far whether 2 < 1 in 
Theorem 1 1.21 and whether Y m and Y k coincide completely when e € (e, 1). It is also not clear whether the 
(J3, e) sub-region in which cr(yp e (2n)) is CS is connected or not. 



8 



This paper is organized as follows. In §2, we give the definitions of w-Maslov-type index to fix notations 
and its relation to o>-Morse index. Some basic variational facts on the elliptic Lagrangian solutions are also 
recalled. In §3, we study the linear stability along the three boundary segments of the (J3, e) rectangle 
[0, 9] X [0, 1). In §4, we prove the hyperbolicity of the elliptic Lagrangian solutions in the case of equal 
masses (Proposition 1 1.41 ) and the non-degeneracy stated in Theorem ll.il In §5, the stability behavior in the 
limit case e —> 1 is considered by the sesquilinear forms of linear operators, and Theorem ll.7l is proved. In 
§6, we investigate the oj degeneracy curves for general a> e U\{1} in the unit circle and establish Theorem 
11.51 In §7, we concentrate on the -1 degeneracy curves. In §8, we study the non-hyperbolic region and 
prove Theorem II .6 1 and the first half of Theorem ll.2l including its items (i)-(iii) and (ix)-(x). Section 9 is on 
the hyperbolic region, and we prove the second half of Theorem II .21 including its items (iv)-(viii) and (xi), 
as well as Theorems 11.81 and 11.91 Finally in the conclusion Section 10, we will give more comparisons for 
results of Martinez, Sama and Simo and our theorems as well as some possible future considerations. 



2 Preliminaries 

2.1 a>Maslov-type indices and a>Morse indices 

Let (R 2n , Q) be the standard sy mplectic vector space with coordinates ( x\ , . . . , x n , y\ , . . . , y n ) and the symplectic 

form O = Y!i=\ dxi A cfyi. Let 7 = 1 " be the standard symplectic matrix, where /„ is the identity 

matrix on R". 

As usual, the symplectic group Sp(2n) is defined by 

Sp(2n) -{Me GL(2«, R) | M T JM = J), 

whose topology is induced from that of R 4,r . For r > we are interested in paths in Sp(2«): 

P T (2n) = {y e C([0, r], Sp(2n)) | y(0) = I 2n }, 

which is equipped with the topology induced from that of Sp(2«). For any a> e U and M e Sp(2n), the 
following real function was introduced in ||9] : 

D^M) = (-1)"" 1 To" det(M - o)I 2n ). 

Thus for any a> e U the following codimension 1 hypersurface in Sp(2«) is defined (191): 

Sp(2rc)° - {Me Sp(2n) | D^M) = 0}. 

For any M e Sp(2n)^, we define a co-orientation of Sp(2«)° at M by the positive direction ^- t Me tJ \ t= o of the 
path Me tJ with < t < e and e being a small enough positive number. Let 

Sp(2< = Sp(2rc) \ Sp(2n)° , 

P* T J2n) = { 7 eP T (2n)\y(T)eSp(2nT w }, 

Plj2n) = P T (2n)\P* Tt(0 (2n). 

For any two continuous paths £ and r\ : [0, t] —> Sp(2?i) with ^(r) = ?7(0), we define their concatenation by: 

1 tW U(2?-t), ifr/2<?<r. 
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Given any two 2m k x Im^ matrices of square block form M k = 



A k B k 



with k - 1,2, the symplectic sum 



of Mi and M 2 is defined (cf. [9J and |QT]]) by the following 2(mi + m 2 ) X 2(mi + m 2 ) matrix M l oM 2 : 



M\oM 2 = 



Mi 









c 2 



Bi 


Ol 




^ 
B 2 


£>2 



(2.1) 



and M ok denotes the k copy o-sum of M. For any two paths jj e P T (2nj) with 7 = and 1, let 70071(f) 
7o(0ori(0forall?e [0,r]. 

As in El, for A e R\ {0}, a e R, 9 e (0,tt) U (tt, 2;r), » - ( f 1 f 2 ) with Z?,- e R for i = 1,...,4, 

c ; - € R for 7 = 1,2, we denote respectively some normal forms by 



and 



D(A) = ( 
Ni(A,a) = 

M 2 (^,c) - 



i 
A~ l 
A a 
A 
(A 1 
A 





R(6) = 
N 2 (e 



cos 
sin 6* 



- sin# 
cos # 



' ' I R(6) 



ci 

-A" 2 





(-A)c 2 


A" 1 



Here N 2 (e ^~ w , b) is trivial if (b 2 -b 3 ) sin > 0, or non-trivial if (» 2 -03) sin < 0, in the sense of Definition 
1.8.11 on p.41 of Cl. Note that by Theorem 1.5.1 on pp.24-25 and (1.4.7)-(1.4.8) on p.18 of HH, when 
A = - 1 there hold 

c 2 t if and only if dimker(M 2 (-l, c) +/) = 1, 
c 2 = if and only if dimker(M 2 (-l, c) + 1) = 2. 

Note that we have N\(A, a) « iVi(/l, a/M) for a e R \ (0) by symplectic coordinate change, because 

I VM / V A)\ 1/ VR / ~ V0 A )■ 
Definition 2.1 f/|9]/, EIB) For any weU and M € Sp(2«), <ie/wie 

v„(M) = dime ker c (M - coI 2n ). (2.2) 

For every M e Sp(2«) and a> e U, as in Definition 1.8.5 on p. 38 of HUH , we define the w-homotopy set 
tojtf) ofM in Sp(2n) by 

Q W (M) = e Sp(2rc) I Voj {N) = v a (M)}, 
and the homotopy set Q(M) of M in Sp(2n) by 

Q(M) = {N e Sp(2«) | o-(iV) n U = cr(M) n U, and 

V/l (A0 - v A (M) V A e cr(M) n U}. 

We denote by QP(M) (or Q^(M)J /raf/i connected component ofQ.(M) (Q. 0) (M)) which contains M, and 
call it the homotopy component (or w-homtopy component) of M in Sp(2«). Following Definition 5.0.1 
on p. Ill of HI IV . for co e U and j\ € !P T (2«) with i - 0, 1, we write jq ~ a) y\ if is homotopic to 
j\ via a homotopy map h e C([0, 1] x [0, t], Sp(2rc)) such that h(0) = y$, h(\) = j\, h(s)(0) - I, and 
h(s)(T) € Q° (yo(r)) for all s e [0, 1]. We write also 70-71. ifKs)(j) e O (y (T)) /or a// s e [0, 1] is further 
satisfied. 
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Following Definition 1.8.9 on p.41 of ifTTI . we call the above matrices D(A), R(8), N\(A, a) and N%(c), b) 
basic normal forms of symplectic matrices. As proved in [|9| and iflOl (cf. Theorem 1.9.3 on p.46 of IfTTI '). 
every M e Sp(2«) has its basic normal form decomposition in QP(M) as a o-sum of these basic normal 
forms. This is very important when we derive basic normal forms for y^^(2n) to compute the aj-index 
ico(7/3,e) of the path jp e later in this paper. 

We define a special continuous symplectic path % n c Sp(2«) by 

/ 2 - - \°" 
Ut) = [ Q T (2 _i)-i) forO<f<r. (2.3) 

Definition 2.2 (M £771/ ) For any t > and y € !P T (2«), <ie/z«<? 

v^y) - vJy(T)). (2.4) 

7/7 e £>* w (2n), define 

4,(7) = [Sp(2n)° :y*&], (2.5) 

where the right hand side of d2.5| ) w ws^a/ homotopy intersection number, and the orientation ofy*£ n is 
its positive time direction under homotopy with fixed end points. 

Ify e 'P^iln), we let T(y) be the set of all open neighborhoods ofy in P T (2n), and define 

ijy) = sup M{L(fl)\p£Un>P* T0J (2n)}. (2.6) 

UeT(y) 

Then 

(; w (r),y w (y))eZx{0,l,...,2n}, 

is called the index function ofy at co. 

We refer to IfTTI for more details on this index theory of symplectic matrix paths and periodic solutions 
of Hamiltonian system. 

For T > 0, suppose x is a critical point of the functional 



F{x)= f L(t,x,x)dt, V xe W l ' 2 (R/TZ,R"), 
Jo 



where L e C 2 ((R/rZ) x R 2 ", R) and satisfies the Legendrian convexity condition L p p (t, x, p) > 0. It is well 
known that x satisfies the corresponding Euler-Lagrangian equation: 

d 

—L p (t, x, x) - L x (t, x, x) = 0, (2.7) 
dt 

x(0) = x(T), x(0) = x(T). (2.8) 



For such an extremal loop, define 



Note that 



P(t) = L PtP {t,x(i),x(i)), 
0(0 = L XiP (t,x(t),x(t)), 
R(t) = L x , x (t,x(t),x(t)). 



F "w = -TS p T f + Q^ + Q T T< +R - ^ 

dt dt dt 
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For ueU, set 

D{oj, T) = {y € W l2 ([0, T], C") | y(T) = wy(0)}. (2- 10) 

We define the oj-Morse index (p^ix) of x to be the dimension of the largest negative definite subspace of 

(F"(x)y u y 2 ), V y u yi * D(oj,T), 

where (•, •) is the inner product in L 2 . For to € U, we also set 

Dip), T) = {ye W 2 < 2 ([0, T], C n )\y(T) = coy(0),y(T) = ajy(0)}. (2.11) 

Then F"(x) is a self-adjoint operator on L 2 ([0, T], R") with domain D(u), T). We also define 

v w (x) = dimker(F"(;c)). 

In general, for a self-adjoint operator A on the Hilbert space J4f, we set v(A) = dimker(A) and denote by 
<p(A) its Morse index which is the maximum dimension of the negative definite subspace of the symmetric 
form (A-, •). Note that the Morse index of A is equal to the total multiplicity of the negative eigenvalues of 
A. 

On the other hand, x(t) = (dhjdxif), x(t)) T is the solution of the corresponding Hamiltonian system of 
(I2.7I) - (I2.8I) . and its fundamental solution y(t) is given by 

y(t) = JB(t)y(t), (2.12) 
7(0) - hn, (2.13) 



with 

m ~\-Q(t) T P-\t) Q(t) T P'\t)Q(t)-R(t)j- {IAV 



Lemma 2.3 (Long, HI IV . p. 172) For the oj-Morse index (p^ix) and nullity v^ix) of the solution x - x(t) and 
the oj-Maslov-type index / w (y) and nullity v w (y) of the symplectic path y corresponding to x, for any u> € U 
we have 

(pa>{x) - icoiy), v w (x) = v w (y). (2.15) 

A generalization of the above lemma to arbitrary boundary conditions is given in Q. For more infor- 
mation on these topics, we refer to ifTTI . 



2.2 Stability criteria via Morse indices 

In this subsection we recall some results of [|5|. 

Gordon's classical theorem (cf. 0) says that the Keplerian solution is a minimizer in the loop space 
under some topological constraint. To the essential part, a theorem of Venturelli in |[T8l as well as Zhang and 
Zhou in |[T9l tells us that the Lagrangian solution is the minimizer among the loops in its some homology 
class. Note that, up to now, these are the only known variational facts under topological constraints on the 
loop spaces. 

By these theorems, we got criteria for the stability in terms of Morse indices. Let <f> k = (p\(q k ) be the 
Morse index of the &-th iteration q k of the Lagrangian solution q in the variational problem, and according 
to ED and 03, 4> l = holds. 

The Lagrangian solution is called linearly stable (spectrally stable) if y(2^) is linearly stable(spectrally 
stable). The first and the third named authors proved the following 
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Theorem 2.4 (Hu-Sun, 15 j) For the monodromy matrix M corresponding to the elliptic Lagrangian solution 
q = (qi(t),q 2 (t),q3(t)), 2 < cf> 2 < 4 and, 



e(M)/2 > 



(2.16) 



Moreover 

(i) If <p 2 — 4, then the Lagrangian solution is spectrally stable; 

(ii) If (f> 2 = 3, then the Lagrangian solution is linearly unstable; 

(Hi) If (j) 1 =2, then the Lagrangian solution is spectrally stable if there exists some integer k > 3, such 
that <p k > 2(k - 1). 

(iv) If (p k = 2{k — I), for all k € N, then the Lagrangian solution is linearly unstable. 

Moreover, if the essential part y = yp fi (i) (cf. Subsection 2.3 below) of the monodromy matrix at t = An 
is non-degenerate, we can get the normal forms of yp, e (2n). 

Theorem 2.5 (Hu-Sun, /0/J In the same setting of the above theorem, suppose yp, e (An) = yp, e (2n) 2 is non- 
degenerate. 

(i) lf(j) 2 = 4, then y p , e {2n) » R(2n - G\)oR(2t: - <9 2 ) holds for some 8\ and 62 £ (0, n), and is linearly 
stable; 

( ii) If '(f> 2 = 3, then yp^ e (2n) « D(A)oR(2n — 9) for some A < and 6 € (0, n), and is linearly unstable; 
(Hi) If(p 2 — 2 and there exists some integer k > 3 such that cp k > 2(k- 1), then yp^ e (2n) x R(2n-6\)oR(d2) 

holds with < 6\ < 82 < n, and is linearly stable; 

(iv)If(f> k = 2(k - I) for all fceN, then yp, e {2n) is hyperbolic or spectrally stable and linearly unstable. 



2.3 The essential part of the fundamental solution of the elliptic Lagrangian orbit 

Following Meyer and Schmidt (cf. p.275 of |[T5l ). the essential part y = y^^if) of the fundamental solution 
of the Lagrangian orbit satisfies 



yit) = JB(t)y(t), 
7(0) = h, 



(2.17) 
(2.18) 



with 



5(0 = 



n 

1 

-1 

1 





-1 

2ecos(7)-l- -0T7j 
2(l+ecos(0) 





where e is the eccentricity, and t is the truly anomaly. 
Let 

'0 -1 



1 




2e cos(f)-l+ y[9~^t 
2(l+ecos(f)) 



/ 3+ a/9 



2(l+e cos(0) 





and set 



1, 



1 



L(t,x,x) = -||i;|| + J2x(t) ■ x(t) + -Kn e (t)x(t) ■ x(t), 



3- ^9-/3 
2(l+ecos(/)) ' 



Vxe W l ' 2 (R/2nZ,R z ), 



(2.19) 



(2.20) 



(2.21) 



where a ■ b denotes the inner product in R 2 . Obviously the origin in the configuration space is a solution of 
the corresponding Euler-Lagrange system. By Legendrian transformation, the corresponding Hamiltonian 
function is 



H(t,z) = ^B(t)z-z, 



V z € R 4 . 
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Note first that the elliptical Lagrangian solution is a local minimizer of the action functional 9K on the 
homology class of curves with winding number (1, 1, 1) or (-1, -1,-1) in H\{X) (cf. lfT8l . lfT9l . and Lemma 
4.1 of @). Note secondly that curves with winding number not equal to ±1 can not approximate curves with 
winding number 1 or - 1 in C(R/Z, R 2 ). Therefore the Morse index of 3\ at the elliptical Lagrangian solution 
in the whole space X and that restricted in the homology class of curves with winding number (1, 1, 1) or 
(-1,-1, -1) in H\{X) coincide and take the value zero. 

Here note that let y\ be the fundamental solution of the Kepler orbit. Then y\ oyp, e is the fundamental 
solution of the Lagrangian orbit. By Theorem 7.3.1 on p. 168 of ifTTI and the additivity of the index theory 
(cf. (ii) of Theorem 6.2.7 on p. 147 of ifTTTO . we obtain 

/ = h{y\) + h(4, e )> v * eN ' ( 2 - 22 ) 

where <p k is defined at the beginning of §2.2. Note that by Proposition 3.6 in p.l 10 of [5], we have 

^(-y*) = 2(k - 1), ViceN. (2.23) 
Thus by our above discussions and (I2.22I )- (I2.23I ) with k - 1 , we obtain 

h(7fi,e) = <P l = 0, V (fi, e) e [0, 9] x [0, 1). (2.24) 

2.4 A modification on the path y^ e (t) 

In order to transform the Lagrangian system (12.191 to a simpler linear operator corresponding to a second 
order Hamiltonian system with the same linear stability as yp^ e {2n), using R(t) and R A {t) = N 2 {e nf ~ l ,0) 
defined in §2.1, we let 

= RdVypAi), V t e [0, 2n], (J3, e) e [0, 9] x [0, 1). (2.25) 
One can show by direct computation that 

dM t} = J ( R(t)(h - K P , e (t))R{t? ) ^ {ty (2 " 26) 

Note that /?4(0) = R A {2n) - I4, so y^^(2n) = t;p^{2n) holds and the linear stabilities of the systems (12.181 ) 
and (12.261 ) are precisely the same. 

By (12.251 ) the symplectic paths yp fi and gp, e are homotopic to each other via the homotopy h(s, t) = 
R4{st)y^ e (i) for (s,t) e [0, 1] x [0,27r]. Because R4{s)y^ e (2n) for s € [0, 1] is a loop in Sp(4) which is 
homotopic to the constant loop yp^{2n), we have ~i by the homotopy h. Then by Lemma 5.2.2 on 
p. 117 of IfTTI . the homotopy between yp, e and ^ e can be realized by a homotopy which fixes the end point 
7y3 je (2^) all the time. Therefore by the homotopy invariance of the Maslov-type index (cf. (i) of Theorem 
6.2.7 on p. 147 of El) we obtain 

hMpJ = io>{jp,e), vMfi,e) = vjy^), V co e U, (J3, e) e [0, 9] x [0, 1). (2.27) 

On the other hand, the first order linear Hamiltonian system (12.261 ) corresponds to the following second 
order linear Hamiltonian system 

x(t) - -x(t) + R(t)K^ e (t)R(t) T x(t). (2.28) 
For (fi, e) e [0, 9) x [0, 1), the second order differential operator corresponding to (12.281 is given by 

A(fi,e) = --^h-h + R(t)Kp,e{W) T 

= ~h ~h + ^-r— -(3/ 2 + y/9=0S(t)), (2.29) 

dt 1 2(1 + <?cos t) 
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where S(t) = ( cos ^ t srn ^ defined on ^ domain D(a>,2n) in < |2.11b . Then it is self-adjoint and 
sin. 2*t cos 2*x i 



depends on the parameters B and e. By Lemma 1231 we have for any B and e, the Morse index w (A(j3, e)) 
and nullity v w (A03, e)) of the operator A 03, e) on the domain D(a), 2n) satisfy 



e)) = ia>{tp, e ), v^Aip, e)) = v^e), VweU. 
Specially by Lemma 4. 1, (55) and (58) of @ and the above (12.24b . we obtain 

h(fy,e) = 0i (AOS, «)) = ii(y^) - 0, V OS, g) e [0, 9] x [0, 1), 



(2.30) 



(2.31) 



In the rest part of this paper, we shall use both of the paths y^ e and ^ e to study the linear stability of 
y/j, e (2n) = g/3 te (2n). Because of (12.271) . in many cases and proofs below, we shall not distinguish these two 
paths. 

3 Stability on the three boundary segments of the rectangle [0, 9] x [0, 1) 

We need more precise information on stabilities and indices of the three boundary segments of the OS, e) 
rectangle [0,9] x [0, 1). 



3.1 The boundary segment {0} x [0, 1) 

When B = 0, this is the case with two zero masses, and the essential part of the fundamental solution of 
Lagrangian orbit is also the fundamental solution of the Keplerian orbits. In fact, when (5-0, without loss 
of generality, by the definition (11.41 ) of B we may assume mi = = and m\ > 0, and then eveiy elliptic 
Lagrangian solution becomes the motion along two Keplerian solutions of the two points qi and with 
zero masses going along their elliptic orbits around fixed point q\{t) = with mass m\ > 0. When /3 = 0, 
the matrix B(t) in (12.191 ) becomes 



5(0 = 



/ 1 








1 \ 





1 


-1 








-1 


2— e cos(t) 





i+e cos(?) 


. 1 








1 ) 



(3.1) 



which coincides with the coefficient matrix B(t) in (17) on p.275 of lfl5l . Note that by the different sign 
choice of the standard symplectic matrices on p.259 of ifTBI . the order of the independent variables there are 
different from that in our system (I2.17l i- (l2.19l l. 
(A) The case of e — 0. 

In this case, the matrix B(t) becomes independent of t: 



B = B(t) 



( 1 








1 \ 





1 


-1 








-1 


-2 





. 1 








1 ) 



(3.2) 



Then one can find the fundamental solution yo,o(0 of the corresponding system (T2.lt with constant coeffi- 
cient JB explicitly: 



7o,o(0 = 



2 - cos t 3t - 2 sin t 3t - sin t 1 - cos t \ 

- sin t 2 cos t — 1 cos t — 1 - sin t 

sin t 2-2 cos t 2 - cos t sin t 

2 cos t — 2 4 sin t - 3t 2 sin t - 3t 2 cos t - I 



(3.3) 
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Letting 



1 1 








6n \ 





-l/(6r) 


-1 











1 





lo 








-6jt) 



we then obtain 



7(0 = P~ l 70fi{t)P 



( cos t 

sin? 

2—2 cos f 
6^ 



_ 2 sin f 
1 

2 cos f— 2 

6tt 
4 sin t-3t 

36n 2 



- sin? 


cos t 

2 sinf 
6tt 



0\ 




1 



Next for e e [0, 1] we consider the following homotopy path y e (t): 

.2 sin t 



7e(t) = 



cos t 


sin? 

2-2 cos t 



-e- 



6n 



1 

2 cos t—2 
e 6n 
4 sin f-3f 



- sin ? 


cos t 

_ 2 sin ( 



0\ 







Then y e (f) € Sp(4) and y E (0) 



fo(0 



and specially we obtain 



6;r 36w 2 6w 

U hold for all ? € R and e € [0, 1]. We have yi(f) = y(f) and 



/cos f 





- sin t 


0\ 





1 








sin f 





cos ? 





I 


4 sin t-3t 
36^ 2 





1 - 



R(t)o 



4 sin f— 3f 
36ff 2 



R(t)o 



1 £ 



y ,o(2?r) = 



1 1 


6/r 





1 








lo 


-67T 



6tt 



1 

-6n 



0\ 





1 



= p 



r l 








0> 





1 














1 





lo 


1 

6tt 





1 . 



/ 2 oiVi(l,l). 



(3.4) 



(B) The case of e € [0, 1). 

Note that it is well known that the matrix <1>(0 with t = f in Lemma 3.1 on p.271 of |[T5l consists of two 
parts, one of which corresponds to the Keplerian elliptic solution of the two-body problem and the other of 
which corresponds to the coefficient matrix B(t) of the essential part Jp, e (t) in our notations. Specially when 
f3 = (i.e., cr = in lfT5l ). both the two parts of the matrix ®(?) coincide to each other, and yields precisely 
(D(0 = B(t)oB(t) there. Therefore when ft = 0, the essential part y J g, f ,(f) corresponds to the Keplerian elliptic 
solution of the two-body problem. 

Thus by Lemma 3.3 and the discussion on (46) of [5], the matrix jQ e {2n) satisfies: 



jQ >e (2n) nI 2 oNi{\, \), Vee[0,l), 



(3.5) 



and it includes ( 13.41 ) as a special case. 
(C) The indices ico(7o,e)for well. 
By (I23TT ) we obtain 

MMO, e)) = h{^ e ) = h(y Q J = 0. (3.6) 

By ( 13.51 ) and properties of splitting numbers in Chapter 9 of IfTTTl . as in (56) of Q, for to € U \ {1} and 
M - yo^(2n) we obtain 

= h{^o,e) + S\f{V) - S~ M {co) 
= + S + (l) + S£ l(W) (l)-0 

= 2. (3.7) 
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For every e e [0, 1), note that (13.51 ) yields also 

V l o(70,e) = V„(€0,e) = {ll tf " U^l}. ^ 

3.2 The boundary segment {9} x [0, 1) 

This is studied in our Proposition 1 1 .41 and more precisely in §4 below. Specially we have 

cr( r9 , e (2/0)cR + \{l}, Vee[0,l) 

and it possesses a pair of double positive real hyperbolic eigenvalues. By 02.311 ), we have i\{yg te ) - for ai l 
e e [0, 1). By our studies in §4.1, the matrix y^ e (2n) possesses always two double positive eigenvalues not 
equal to 1, thus by the definition of the splitting number in §9. 1 of ifTTTl . we have S^iaj) = for M = y 9 e (2/r) 
with e e [0, 1) and all a> eU. Then for all e e [0, 1) and a> € U this yields 

iw(7%e) = z'w(r9,o) - 0, v w (7 9>e ) - v CJ {y 9fl ) = 0. (3.9) 



3.3 The boundary segment [0, 9] x {0} 



In this case e = 0. It is considered in (A) of Subsection 3. 1 when (3 = 0. When B € (0, 9], this is the case of 
circular orbits with three positive masses. It was studied in Section 4 of lfl6l by Roberts and in pp.275-276 
of lfT5l by Meyer and Schmidt. Below, we shall first recall the properties of eigenvalues of ypjtfkt). Then 
we carry out the computations of normal forms of ypfl(2n), and +1 indices i±\(yp$) of the path yp$ for all 
P € [0, 9], which are new. 

In this case, the essential part of the motion (I2.17l )- (|2.19l ) becomes an ODE system with constant coeffi- 
cients: 



-1 



B = 5(f) 



fl 
1 

-1 

.10 

The characteristic polynomial det(/B - AI) of JB is given by 

A 4 + A 2 + ?-=0. 



1 





~B-\ 



(3.10) 



(3.11) 



Letting a = A 2 , the two roots of the quadratic polynomial a 2 + a + | are given by a = ± yjl -B). 
Therefore the four roots of the polynomial (13.1 11 1 are given by 



(3.12) 



(A) Eigenvalues of y^ Q {2n) for B € [0, 9]. 

When < B < 1, by (13 - 1 2b . we get the four well-known characteristic multipliers of the matrix y/3$(2n) 



PU±(P) = e 2 ™>-- = e ±2 ^W\ 



for i = 1,2, 



(3.13) 



where 



em = ^I(i- vr^sx e 2 (j3) = ^I(i+ vr^6). 



(3.14) 
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When 1 < p < 9, from (13.121 ) by direct computation the four characteristic multipliers of the matrix 
ypft(2n) are given by 

p±± = e ±n yRp~\ e ±* V^T V#T7 (3 15) 

Specially, we obtain the following results: 

When y6 = 0, we have o-{yo${2n)) = {1,1, 1, 1), which coincides with (13.5I ). 

When < P < 3/4, in (13.141 ) the angle 9\(Ji) increases strictly from to 1/2 as/? increases from to 
3/4. Therefore pi, + (/?) = e 2n ^~^ 61 ^ runs from 1 to -1 counterclockwise along the upper semi-unit circle in 
the complex plane C as /? increases from to 3/4. Correspondingly pi _(/?) - e- 2 *"V-r0i(© runs f rom l to -1 
clockwise along the lower semi-unit circle in C as p increases from to 3 /4. At the same time, because 02iP) 
decreases strictly from 1 to V3/2 as p increases from to 3/4, therefore P2,+(P) = e 2n ^~^ ei ^ runs from 1 
to g^-i c i oc k w j se along the lower semi-unit circle in C as p increases from to 3/4. Correspondingly 
P2,-(P) = e~ 2n ^ 02 ^ runs from 1 to g-V^TVlTr counterclockwise along the upper semi-unit circle in C as fi 
increases from to 3/4. Thus specially we obtain cr(7/? ,o(2?r)) c U \ R for all/? e (0, 3/4). 

When p = 3/4, we have ft (3/4) = 1/2 and ft(3/4) = V3/2. Therefore we obtain pi,±(3/4) = e ±VZLr = 
-1 andp 2 , ± (3/4) ^ e ± ^~ l ^ n . 

When 3/4 < /? < 1, the angle ft (8) increases strictly from 1/2 to V2/2 as /? increase from 3/4 to 1. 
Thus pi,+(yS) = e^V-fft^) runs f rom _i to gV^TVI/r counterclockwise along the lower semi-unit circle in C 
as fi increases from 3/4 to 1. Correspondingly pi -OS) = g-^V-ieiO) rU ns from -1 to ^-V-TV^r clockwise 
along the upper semi-unit circle in C as fi increases from 3/4 to 1. Because ft (8) decreases strictly from 
V3/2 to V2/2 as fi increases from 3/4 to 1, we obtain that p 2 ,+(P) = e 2 "^ 101 ^ runs from e^^ n to 
eV-iVSw clockwise along the lower semi unit circle in C as fi increases from 3/4 to 1. Correspondingly 
P2,-(P) = e^ 2 "^ 62 ^ runs from e -V-i V3tt t0 g - V^TVItt counterclockwise along the upper semi unit circle in 
C as p increases from 3/4 to 1. Thus we obtain cr(yp fi (2n)) c U \ R for all p e (3/4, 1). 

When p = 1, we obtain ft(l) = ft(l) = V2/2, and then we have double eigenvalues pi,±(l) = P2,±(l) = 

When 1 < p < 9, using notations denned in (13.151) . the four characteristic multipliers of ypflQjt) satisfy 
o-(7 j8 ,o(27r)) c C \ (U U R) for all p e (1,9). 

When p = 9, J V9 + In = 2n. By (13.151) . we get the two positive double characteristic multipliers of 

79,o(2tt) given by p ± , ± = e ± " j2n e ± ^~ i2n = e ± ^ 2n € R + \ { 1 }, where we denote by R + = [r € R | r > 0). 
(B) Normal forms of yp fi (2ji) for p € [0, 9]. 

By our above analysis, the matrix ypfl(2n) possesses no eigenvalues ±1 for p e (3/4,9]. Note also that 
the matrix ypfl{2n) is homotopic to N2((Oo,b) with ojq - e^^"^ and b 2 - b-$ ^0 when p - 1, and is 
hyperbolic when p e (1, 9]. Therefore by Definition 12. 11 we have 

Jpfl ~±i 7,60,0, V/3 e [1,9] and p Q e (3/4, 1). (3.16) 

Thus next we are specially interested in the normal forms of ypfl(2n) with p € [0, 1). 

For that purpose we construct a family of continuous curves f(P){t) in Sp(4) for p e [0, 1) and t e [0, 2n] 
satisfying f(P)(0) = I such that y^{2n) ~ ±1 f(P) for all p e [0, 1), which implies ypfi(2n) » f(P) for all 
P € [0, 1). After that we shall study the normal forms of yp${2n) for p € [1,9]. 

The curve / is defined separately according to p as follows. 

(i) Normal forms ofypfi{2n) when p € [0, 3/4]. 

Here, we splits / into a symplectic sum of two Sp(2)-paths: 

f(p) = f + (p)of-(p), f ± (P) 6 Sp(2). (3.17) 
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Using the cylindrical coordinate representation (which is denoted by CCR for short below) of Sp(2) 
introduced in (H (cf. pp.48-50 of ifTTl . specially Figure 1 and 2 on pp.49-50 there), we can describe the 
matrix curves in Sp(2) more precisely, which is shown below in Figure 3. 

Using notations in §2 we let 

8/5 6/5' 



M -D(2)7?(27r-arcsin(3/5))-(_ 3 / /10 (3.18) 



Then M w N^l, 1) and thus M e Sp(2)° _ n {(r, 6,z) s R 3 \ {z-axis} | z = 0} in Figure 2.1.2 on p.50 of 
Ifm . By 03.31 ) and our discussions in part (A), we define 

/ + (0)=/ 2 and f-(0) = M . (3.19) 

By our analysis in part (A), when fi increases from to 3/4 in (0, 3/4), we define 

f + (J3)=R(2n-^nfi), for fi e [0,3/4]. (3.20) 

Then the matrix curve f+iP) runs from I2 to R(n) = -I2 along the left semi-circle clockwise in CCR in the 
left diagram of Figure 3 below. 

Then we can choose some fi\ € (0, 3/4) and define 

/_08)=D(2-|-)/?((27r-arcsrn(3/5))(l-|-) + for £ e [0,pU (3.21) 

Pi Pi j 



and 

_ 3/4-/3 i-9 
/_(£) = R( V3tt - ; p ( V3 - -);r), for ft € \fi u 3/4]. 
3/4 -pi 5 



(3.22) 



Thus the matrix curve f~{fi) runs from /_(0) = Mo to f-{fi\) = R(9n/5) when fi runs from to fi\, and then 
runs from /_(jSi) = R(9n/5) to /_(3/4) = R( V3;r) when fi runs from fi x to 3/4. The image of f-(fi) is shown 
along the left semi-circle clockwise in CCR in the right diagram of Figure 3 below. 

Then by adjusting the running speeds of f + (fi) and f-(fi) according to those of the corresponding eigen- 
values in ( 13.131 ) respectively, we can have 

f(fi) - f+(J3)of-(fi) * r J e,o( 27r ). for jB 6 [0,3/4]. (3.23) 
Specially when fi = 3/4, we obtain 

/(3/4) - -I 2 oR( V3tt). (3.24) 

(ii) Normal forms ofjpfl{2n) when fi € [3/4, 1). 
For fi € [3/4, 1), following part (A), we define 

f + {fi) = fl(4(l- V2)7Z)8 + (3V2-2>r), (3.25) 
/-OS) - «(4( V2 - V3)tt/3 + (4 V3 - 3 V2». (3.26) 

By adjusting the two curves f+(Ji) and /_(j3) suitably when fi < I and close to 1 and then adjusting their 
running speeds in fi suitably, we can suppose that when fi increases from 3/4 to 1 in (3/4, 1), the matrix 
curve f+(fi) runs from -I2 and tends to R((2 - V2)7r) along the right semi-circle clockwise in CCR in the 
left diagram of Figure 3 below as fi runs from 3/4 and tends to 1. Simultaneously the matrix curve f-ifi) 
runs from R{ V37r) and tends to R{ ^lln) along the left semi-circle clockwise in CCR in the right diagram of 
Figure 3 below as fi runs from 3/4 and tends to 1. Here we shall not explain how f(fi) = f+(Ji)of-(Ji) gets to 
its limit when fi — > 1. Note that in this case we have also (13.231 ) holds when fi e [3/4, 1). 
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(iii) Note that for ft € [0, 1), we have f+(J3) = R{a(J3)), where the value of a(J3) is uniquely given by 
(13.201 ) and (13.25 1 Then every matrix f(J3) = f+(/3)of-(J3) e Sp(4) in (13.171 ) can be reached by a path starting 
from I4 in Sp(4) as follows: 



f + (fi)(i) = R{a(fi)—\ 
2n 



for < t < 2n, 



(3.27) 
(3.28) 



( D(2t/n)R((2n - arcsin(3/5))f/7r), if <t <n, 
I f-W^X if n < t < In, 

for j6 € [0, 1), where we have used the expression of the matrix Mq in d3.18| ). 
(iv) The normal form ofypfl{2n) when ft = 1. 

Firstly by Corollary 14. 5 1 below, for fixed e e [0, 1) the index ia){yp,e{2n)) is non-increasing when [3 
increases from to 9 for a> e U \ {1}. By Proposition 16. 1 1 below, the index ioj{yp, e {2n)) can change only 
when the matrix path yp iS in [3 passes through an u degeneracy point and there should be either one or two 
oj-degeneracy points and their total co degenerate multiplicity is 2. 

By part (A), ft - 1 is a Krein collision point of the matrix path yp$Q-n) for ft e [0, 9] with o-(y\${2n)) = 
{ojq, (Do, wo,wo) for u>o - e ^~^^ n € U. Note that by our above discussions, when yS increases in the open 
interval (0,3/4), the curve f+(fi) passes through each a> singular surface in Sp(2) precisely once for all 
oj e U \ R, which contributes precisely a 1 to li/je[o,9] v u(yp,o(2n)). Therefore by Proposition 16. 1 I below we 
must have 



(yi fi (2n)) = dim c ker c (ri,o(27r) - w 7) = 1. 



Then, we must have 

7\,o(2n) 



N 2 (oj ,b) = ( 



R(^/2n) 







/?(V2tt) 



with b - 



b 2 
bx 



(3.29) 



(3.30) 



satisfying b 2 - b^ + 0. 

(v) Normal forms ofypft{2n) when /3 6 (1, 9]. 
For p e (1, 9), by part (A), we have 



e*^~ l R(nJ~jjf+ 1) 

e- n ^P~ l R(nyj^+ 1), 

for some matrix b = b(fj), which is CS-hyperbolic. When f3 - 9 we get 

79,0(20 



D(e^)«D(e^). 



(3.31) 



(3.32) 



Remark 3.1 Because B{t) is a constant matrix depending only on [3 when e = 0, similarly to what we did 
for yo,o(0 it is possible to compute out the fundamental matrix path ypfiif) explicitly when fi > 0. But the 
computations on yp${t) when > are rather delicate and tedious and thus are omitted here. From this 
computation, specially when < /? < 1 we obtain that y/?,o(0 ~ R(-2n9i(fj)t)oR(2n82(p)t) for some #,(/?) 
with i = 1,2, and the /3-paths R{-2n6\{fi)) and R(2n02(fJ)) are homotopic respectively to f+(J3) and f-(fJ) 
which we constructed above. 



(C) Indices z+Ky/j.o) for ,8 e [0,9]. 

From the above discussions as well as Figure 3 we obtain 



h(f + (J3)) - 1, 



vi (/+(£)) = 



vi (/-OS)) 



2. 

,0, 

h(f-(J3)) = -l, 
1, 
0, 



VjS e [0, 1), 
if P = 0, 

if /3 e (0, 1). 
VySe [0,1), 
if /3 = 0, 

if yS e (0, 1). 



(3.33) 
(3.34) 
(3.35) 
(3.36) 
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Figure 3: The pictures of the two paths f+(J3) and f-(J5) in Sp(2), where the three dark points are their images 
aty3 = 0,3/4 and 1. 



Therefore by (13.171 ) we get 

ii(rA0) = 0, V/3e[0,9], (3.37) 

J 3, ifj8 = 0, 

yi( ^ 0) = i0, if (0,9]. (3 " 38) 

Similarly for the -1 index, we obtain 

M(/ + ^))-| 0; if ^ [3/4)1) _ (3-39) 

/°- if e [0, 1) \ {3/4}, ..... 

v-i(f + W)) = { 2i if ^ = 3/4. (3 " 40) 

i-iC/-C8)) = 0, v_i(/_C8)) = 0, if fi e [0, 1). (3.41) 

Therefore by (13.161 ) and Proposition 16. 1 I below. we get 

. /2, if £ e [0,3/4), 

M(r/W) = { 0t if y6e p/4,9]. (3 " 42) 

JO, if /? 6 [0,9] \ {3/4}, ..... 

v-i(7 A o) = ( 2j if ^ = 3/4. (3 " 43) 

The other w-indices of y^o can be computed similarly for u> + e ±^f-^^ = ajQ) an( j /^(y^o) can be 
computed using the decreasing property of the index proved in Corollary 4.5 and its values at /3 = and 
ft = 9 as we did in the proof of Theorem 1 1 .2 I below. 

Here we point out specially that /? = 1 is the only Krein collision point on the segment [0, 9] x {0}. 

4 Non-degeneracy of elliptic Lagrangian solutions 

Note that the complete eigenvalue 1 non-degeneracy implies that there is no linear stability change near the 
positive real line in the complex plane C. 

4.1 Hyperbolicity of elliptic Lagrangian solutions with equal masses 

In the equal mass case, that is /? = 9, we have 

dt 1 2( 1 + e cos t) 
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Let 

d 2 3 

A ^ = ~iz ~ 1 + im 7v < 4 - 2) 

dt 1 2(1 + <?cosf) 

which is a self-adjoint operator with domain D(l,27r) c IV 2 ' 2 ([0, 2n], R) under the periodic boundary con- 
ditions x(t) = x{t + 2n), x(t) = x(t + 2k) for all t € R. Then 

A(9,e)=Ai(e)©Ai(e). (4.3) 

By (12.311) . the Morse index of A (9, e) is zero, so A\{e) is a non-negative operator. Moreover, we have 

Proposition 4.1 A\{e) > for all < e < \ on its domain D(\,2n). 

Proof. It suffices to show ker(Ai(e)) = {0} for all < e < 1. We argue by contradiction. Suppose 
£ x e ker(Ai(e)) is expressed as a Fourier series x = x{t) - 2ZkeZ a k e x P( V-lkt). Then we have 

- 2(1 + ecosi)Ai(e)x(i) 

= (2 + e exp( V-ff) + e exp(- V-ff)) ^ ^(fc 2 - 1) exp( V-Lfo) + Y 3a* exp( V^TA:?) 

= Y^a^A; 2 - 1) + eajt_i((Jfc - l) 2 - 1) + <?a*+i((it + l) 2 - 1) + 3a /t )exp( V-lfa)- (4.4) 

fceZ 

This implies 

2a k (k 2 - 1) + ea k -i{{k - l) 2 - 1) + ea k+l ((k + l) 2 - 1) + 3a* = 

holds for every k e Z. Let & = 0, we have ao = 0. So * belongs to the subspace V which is spanned by 
{exp ^lkt,k + 0). Note that (-Jjj - l)|y > and 2 (i+ecos t) > ®> so ^i( e ) i s positive on V, which then 
implies that x must be zero. This contradiction completes the proof. | 

Remark 4.2 For e = 1, the operator Ai(e) is singular. By the same argument as above one can show that 
Ai(l)* = also implies that x = in L 2 ([0, 2n], R). 

Proof of Proposition 11.41 Let | e (f) be the fundamental solution of the first order linear Hamiltonian 
system corresponding to A\{e). Then it satisfies 

^■Ut) = h{\ ° 3 k(Q, (4.5) 

VU 1 2 (l+ecos0 / 

1(0) = / 2 . (4-6) 



Thus we have 



and so 



= 6(f), (4-7) 



= 2/J&), v w (ft >£ (27r)) = 2v w (&(2»r)). (4.8) 

By Proposition 14. II we have 

dimker(£,(27r) - 7 2 ) = 0, Vee [0, 1). (4.9) 

Since £ e (2n) is a 2 x 2 symplectic matrix, its eigenvalues are in pair [A, /T 1 } with A real and A + 1 or 
/I e U\{1}. Because z'i(£ e ) = by Proposition 4.1, the matrix % e (2n) must have normal form j with 

/I > or |^ ^ j by fTTj (cf. pp. 179- 183 there). By (14.9b . 1 can not be an eigenvalue of g e (2n) for any 
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e € [0, 1), so the only possible case is that g e (27r) has a pair of positive real eigenvalues not equal to 1, that 
is A and A > and A £ 1, and the eigenvalues of f e (27r) depend continuously on e. So the eigenvalues of 
f e (27r) are real positive numbers not equal to 1 for any e € [0, 1). Specially from (I4.71 i with t = 2n, (14.81 ) and 
the above discussions, we obtain 



iJ&J = 2^(6) = 0, v w (&, e (27T)) - 2v w &(27r)) = 0. (4. 10) 

Thus by (12.271 ) and (12.301 ). the Proposition [L4]is proved. | 
Corollary 4.3 A(9, e) is a positive operator for any u> boundary conditions. 



Proof. When /? = 9, since i\Q e ) = and ^ e (2n) is hyperbolic as we proved above, we have i w (f e ) = 
for any a> € U and e € [0, 1). Thus the a> Morse index 4X£9,e) = £lJ (A(9,e)) = 0, that is, A(9,e) > in 
any a» boundary conditions. Then A(9, e) > follows from the fact that A(9, e) is non-degenerate at any a> 
boundary conditions by ( I4.10I ). | 



4.2 The non-degeneracy of elliptic Lagrangian solutions for to = 1 

For (B, e) e [0, 9) x [0, 1), let A(B, e) = Using (12291 we can rewrite A(fi, e) as follows 

J9-" 



A(fi,e)=A{9,e)+ 5(0= V^l^p^ + on f - t ] = ^pA(fi,e). (4.11) 
2(l+ecos?) yj<)-p 2(l+<?cosf), 

Then we have 

<f>„(A(fi,e)) = 4>JMP,e)\ (4.12) 

v w CA08,e)) = v w (A(jS,e)). (4.13) 

Following from Corollary 14.31 i.e., the fact that A(9, e) is positive definite for any co boundary condition, we 
get the following important lemma: 

Lemma 4.4 ( i) For each fixed e € [0, 1 ), the operator A(fi, e) is increasing with respect to /3 € [0,9) for any 
fixed well. Specially 

|^ e,l "* = 2(9^ A(9 ' e) ' (414) 

for ft e [0, 9) /s a positive definite operator. 

(ii) For every eigenvalue Ap = ofA(fio, eo) with a> e U for some (J3q, eo) e [0, 9) x [0, 1), there holds 

jpWlHh >0. (4-15) 

Proof. It suffices to prove (ii). Let xo = xo(t) with unit norm such that 

A(J3 ,e )x = 0. (4.16) 

Fix eo- Then A(J3, eo) is an analytic path of strictly increasing self-adjoint operators with respect to /?. 
Following Kato ([6j, p. 120 and p.386), we can choose a smooth path of unit norm eigenvectors xp with 
x p = x o belonging to a smooth path of real eigenvalues Ap of the self-adjoint operator A{f3, eo) on D(a>, In) 
such that for small enough 1/3 - Pol, we have 

A(fi, e Q )xp - Apxp, (4.17) 
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where Ap = 0. Taking inner product with xp on both sides of (14.171 ) and then differentiating it with respect 
to p at fio, we get 



where the second equality follows from (14- 16b . the last equality follows from the definition of Aifi, e) and 
(14. Ill ), the last inequality follows from the positive definiteness of A(9, e) given by Corollary 4.3, and the 
fact xq t 0. Thus (14.151 ) is proved. | 
Consequently we arrive at 

Corollary 4.5 For every fixed e e [0,1) and co e U, the index function cp^Ai/J, e)), and consequently 
iaj(y/3, e ), is non-increasing in fi e [0, 9]. When to e U \ {1}, it decreases from 2 to 0. 

Proof. For < fi\ < < 9 and fixed e e [0, 1), when ft increases from fi\ to /?2, it is possible that 
negative eigenvalues of A(fii, e) pass through to become positive ones of A(fi2, e), but it is impossible that 
positive eigenvalues of Aifij, e) pass through to become negative by (ii) of Lemma 1441 Therefore the first 
claim holds. The second claim follows from (13.71 . (13.91 ) and Corollary 14.31 | 

From now on in this section, we will focus on the case of co = 1. Since (pi(A(J3, e)) - 0, we have 
A(J3, <?) > for (J3, e) e [0, 9) x [0, 1). Furthermore, we have 

Proposition 4.6 A(fi,e) > Ofor all (fi,e) e (0,9) x [0, 1) under the periodic boundary conditions, i.e., on 
D(\,2n). 

Proof. It suffices to prove A(J3, e) > 0. This is essentially due to the fact that ker(A(/3, e)) = {0} on 
ifi, e) e (0,9) x [0, 1). In fact, otherwise, there exists an xo = xo(t) with unit norm such that (14.161 ) holds. 
Then (14.171) implies that there exists a negative eigenvalue of A(J3, eo) when ft < /?o is sufficiently close to 
fio, which contradicts to A{f3, e) > 0. | 

Proof ofTheorem ll.il Since dimker(A(/?, e)) - dimker(y ( 8 e(27r) - ^4), we have proved that the elliptic 
Lagrangian solutions are all non-degenerate on (fj, e) € (0, 9] x ([0, 1). It is degenerate when /3 = 0, by (13 - 8b - 
The proof is complete. | 

5 The limiting case e — > 1 

We shall use the sesquilinear form to study the case when e — > 1, and please refer to Chapter 6 of [6] for the 
details on the sesquilinear form. 

In this section we shall deal with the limiting cases e = 1 and -1, then the term (1 + ecos?) in the 
denominator of the expression of the operator A{f3, e) and the corresponding functionals will become zero 
when t - (2k + \)n if e = 1 or 2kn if e — — 1 with k € Z. Note that the boundary condition x(27r) - cox(0) 
is equivalent to the boundary condition xit + 2n) = cox(t) for all t e R in the domain D(co, 2k) with co € U 
of the corresponding functionals. Therefore in order to move the singular times to the end of the boundary 
points of the integral intervals, in this section we use the interval ®(e) = [—n, n] to replace [0, 27r] when we 
study the case e — > 1, and keep 0(e) = [0, 27r] when we study the case e — > -1 for all the corresponding 
integrals. 



(—A(fiQ,eo)XQ,XQ) 



> 
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For any (fj, e) e [0, 9] x (-1, 1) and to e U, we define the symmetric sesquilinear form T(J3, e) corre- 
sponding to the operator A{fl, e) in (12.291 ) by 



T(J3,e)(x 



J&(e) 



x ■ y - x ■ y + 



i 



2(1 + ecosO 



((3/ 2 + y[9^pS(t))x(t))-y(t) 



dt, V x,y € D(a>,27r). 

(5.1) 

where the domain D(o», 27r) is defined in (12- 10b . Then we denote by T(J3, e)(x) = T(J3, e)(x, x) the correspond- 
ing quadratic form. Note that here we have extended the range of e to (-1, 1), even to the whole complex 
plan in the next section. This seems artificial from the point of view of celestial mechanics, however we can 
draw some interesting conclusions on the degeneracy curves in the ifi, e) rectangle [0, 9] x (- 1 , 1) later. Note 
that our results proved for e e [0, 1) in the previous and later sections hold also for e € (-1, 1), which we 
shall not specially indicate out explicitly later. Since T(J3, e) + I is equivalent to T(J3, e) with respect to the 
W ' -norm, T(J3, e) is closed on domain D(co, 2n) (O, p.313). 



For e = +1, we further define Y(fi, e) as in (15.1b . but its domain needs to be modified. More precisely, 
say, for e = 1, the domain of T(J3, 1) is defined by 



D(j3) 



xe W 1 ' 2 ([-^,^],R 2 ) 



f \^r^ ^« 3/ 2 + (t))x(t)) ■ x{t)) dt < oo 1 . (5.2) 

J_^\2(l+cos0 / J 



Since 3I 2 + ^(9-/3)S(t) > for < J3 < 9, we have D(J3) = D(9) for < yS < 9 and D(0) is different from 
them. Note that every x e D{9) must satisfy x(n) - x{-n) = which is the vanishing co boundaiy condition. 
And T(fj, 1) is closed since it is the sum of two closed symmetric forms. 

For e e (-1, 1), A(J3, e) is the Friedrichs extension operator ([6 1, Theorem 2.1 in p. 322) of T(J3, e) under 
the co boundary condition. Let A(fi) be the Friedrichs extension operator of T(J3, 1). Then it has the form 



^ ) = -^ /2 - /2 + 2TTTc^) 



h + 



(5.3) 



2(1 + cos t) 
with dom(A(/J)) c D(J3). 

Lemma 5.1 For each ft € (0, 9], A{fJ) is a self-adjoint operator on L 2 ([-n, n], R 2 ) with compact resolvent. 

Proof. Since Y{f3, 1) is symmetric, A(J3) is self-adjoint. It suffices to prove that A{fi) + 2/ 2 has a compact 
erse. Let Y'{x) = Y(J3, l)(x) + 2|W|^,. Tl 
second representation theorem (B, p.331), 



inverse. Let T'(x) - T(/3, \){x) + 2||x|| 2 vl2 . Then A(J3) + 2/ 2 is the Friedrichs extension operator of P. By the 



T'(x) = ((A(J3) + 2/ 2 )i;c, (A(fi) + 2hT-x). 



(5.4) 



Since ||x|| 2 ia < T'(x), a T'-bounded set must have a convergent subsequence in L 2 . This implies that 



1 9 

(A(fi) + 2/ 2 )~2 is compact, so (A(J3) + 2/ 2 ) is also compact in L . | 
Lemma 5.2 For ft e (0, 9] and x € D(fi), we have T(J3, e){x) —> T(J3, l)(x) as e 



1. 



Proof. Let x e D(f3) c D(co, 2n) and e e [0, 1). By definitions of T(f3, e) and T(f3, 1), when cos t > 0, we 
obtain 



1 



1 



2(l+ecosf) 2(1+ cos?) 



)((3/ 2 + y/9^pS(t))x(t)-x(t)) 



(1 - e)cos? 



: ((3/ 2 + V9^5(owo-x(o) 



(1 + ecos ?)(1 + cos t) 
1 - «) l ...A Oh + y/9^fiS(t))x(t) ■ x{t)) 



2(1 + cosO 
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When cos t < 0, we have 
1 



2(1 + ecos t) 
Therefore we get 
1 



((3/ 2 + y/9^pS{t))x(t)-x(t)) 



1 



2(1 + cost) 



2(1 + ecos t) 



((3/ 2 + y/9^S(t))x(t)-x(t)) 



<(2-e) 



1 



2(1 + cos t) 



((3/ 2 + V9^8S(/))x(f) •*(*)) 



((3/ 2 + ,/9=fiS<ty)x(!)-x(fii) 



, (5.5) 



for all f € [0, 2/r]. Now the lemma follows from the Lebesgue's dominated convergence theorem. | 

Since under the periodic boundary condition, T(J3,e) > for e € [0, 1), the above lemma tells us that 
T(J3, 1) > 0. By Remark |4~2l we have ker(A(9, 1)) = {0}, so T(9, 1) > 0. By completely the same reasoning 
as in Proposition 14.61 we have 

IW)>0, V/3 e(0,9]. (5.6) 

For the limiting case e — > 1 under general boundary conditions, we consider the sesquilinear form 
t(J3,e), for/3 e [0,9] and e e (-1, 1), by 



T(j3,e)(x 



•J-n 



| x ■ y - x ■ y + ( — ; x(t) ■ y(t)) 



dt, Vx,y £W 12 (R/2ttZ,R 2 ). (5.7) 



2(1 + ecos?) 
We have the following 

Lemma 5.3 For ft e (0, 9], we have f (6, e) > when 1 - e is small enough. 

Proof. Let 5(fi,e) be the largest lower bound of the quadratic form t(fi,e) for e e [0, 1]. Then by (15.6 
we have 

(50S,1)>0 V0 e(0,9]. 



We need to show 



For <J3, e) e (0, 9] x [0, 1), we define 



liminf 6(B,e) > 0. 

e-»l 



2(l+ecosf) 
2(l+cosf) 



if cos ? < 
if cos ? > 



and 



Let 508, e) be the largest lower bound of f 08, e) on W u (R/(2ttZ), R 2 ) for e e [0, 1]. 
Then by (15.51 ) we obtain 



So it suffices to prove 
For <?2 > e\, we have 



lim inf 506, e) = lim inf 5(fi, e). 

e—* 1 e— > 1 



liminf 5(y6, e) > 0. 



(5.8) 
(5.9) 

(5.10) 

(5-11) 

(5.12) 
(5.13) 



f06,e 2 )«-f06,ei)(*)= f + f / ( g 2 gi)( cos?) + ^— ^ \ ^ 

J-n \{l + e 2 cos t){\ + ei cost) ) 
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Note that 3/ 2 + yfi - /3S(t) is positive definite whenever j3 e (0, 9]. Thus we obtain 

f^,e 2 )>fCS,ei), if e 1 >e 1 . (5.14) 
So 6(J3, e) is increasing with respect to e. Let 

1 



E(fi, e) = \ x e W u (R/(2nZ), R ) 



*ii w ,,2 < 1, rc8,e)(jc) < -<mi)iw£ 2 !>■ 



Then £"06, e) is closed in the Hilbert space W 1,2 (R/2wZ, R 2 ) and £(jS, e 2 ) c £(/3, ei) if e 2 > e\. Let £"06, 1) = 
flo< e <i E{fi, e). We claim 

£G8,1) = {0}. (5.16) 

In fact, otherwise, there exists some x e E(J3, 1) \ {0}. We consider two cases depending on whether 
x € D(f3) or not. If x e D(fi), then f (f3, e)(x) converges to F(f3, l)(x) when e — > 1 by an argument similar to 
that of Lemma [5721 Therefore by the definition of 6{f3, 1) we obtain 

f(fi,e)(x) > y(J3,\)\\x\\ 2 , 

when 1 - e is small enough. This contradicts to the definition (I5.15l l. On the other hand, x e E(fi, 1) \ D(j3) 
implies that 

'3/2+ yj9=pS{t) 



J 



-x(0 • x(0 <i? 



2(1 + cosf) 

is infinite, which contradicts to the definitions of Y(J3, e) and T(/3, 1) as well as Levi's theorem. Thus (15.161 ) 
holds. 

We then further claim that there exists a constant eo e (0, 1) such that 

E(J3,e) = {0], whenever e > e Q . (5.17) 

In fact, otherwise, there exists an increasing sequence e\ € (0, 1) with ieN such that e\ — > 1 and there 
exist Xk e E(fi,ek) with = 1. Since Xk is bounded in W l ' 2 (R/(2nZ), R 2 ), it has a weakly convergent 

subsequence x Hk , which converges weakly to some xq. Then we have xq e E(J5, 1). In fact, by the weakly 
lower semi-continuity of norms, we obtain 

\\xq\\ w i.i < liminf 

On the other hand, by the Sobolev compact embedding theorem, x nk converges to xo in the L 2 space. By 
definition, x e Eifi, e) is equivalent to 

IMIww + J (fpAtMO ■ x{t))dt < (^OS, 1) + 2)N| 2 2 . (5. 18) 



Thus xq € E(J3,e, Jk ) for every implies xq € E(J5, 1). On the other hand, (15.181 ) implies that the lower 

bound of ||x„J| 2 2 is nonzero. So we have xq + 0, which contradicts to ( 15.161 ), and proves (|5.17| i. 

Now by (gl]), (f57T4b and d5TT6l ), for fixed j3 e (0, 1] and every e > e we obtain 5(fi, e) > 6(J3, l)/2 > 0, 
which completes the proof. | 

Since D(a>, In) c IV 1 ' 2 (R/2ttZ, C") for any well, then t(fi, e) > implies T(J3, e) > for any weU. 
Thus we have 

Corollary 5.4 For any fixed [3 € (0,9], there exists an e* e (0, 1) such that for any to € U there holds 
T(j3,e)>0foraUee [e*,l]. 

Proof of Theorem 11.71 The proof of the limiting case e — > -1 is similar and thus is omitted. The above 
lemmas imply that for any fixed /? € (0, 9], we have always A(J3, e) > for any to e U whenever 1 - \e\ is 
small enough. Thus Theorem [L7]is proved. | 
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6 The oj degeneracy curves of elliptic Lagrangian solutions 

For any oj boundary condition, that is on domain D(oj, 2n) of (12.111 ). A(fi, e) is a closed unbounded operator. 
If we extend e to the complex plan and denote the open unit disc by D - \e e C | \e\ < 1}, then A(fJ, e) is 
holomorphic with respect to e € D (O, p.366). It satisfies Aifi, e) = A(J3, ef . In fact 

d 2 1 

Aifi, e) = -h -h + -z (3/ 2 + <J9-fiS(fi)(l - ecos(f) + e 2 cos 2 (f) - e 3 cos 3 (f) + •••), (6.1) 

at 1 2 v ' 

where we have 

3/ 2 + ^9-J3S(t) > 0, for <yS < 9, 

3/ 2 + V9-#S(f) > 0, for <>S < 9. 

Let O be a small narrow neighborhood of the interval (-1, 1) in the complex plan. For e e Q, if its imaginary 
part Ime is small enough, A(9,e) is strictly accretive ([6], p.281), i.e., there exists 8 > such that the real 
part 

Re(A(9,e)x,x) > 6\\x\\ 2 

for any x in the domain D(a>, 2k) of A(9, e). We get A(9, by the Dunford-Taylor integral (|6], (3.43) in 
p.282), 

i i 

A(9,eT? = -\ iu~HA(9,e)+iu)' l diu, (6.2) 
n Jo 

which is bounded and holomorphic in e (O, p. 398). 

By the idea of the proof of Proposition 14.61 we have the following important results for all a> € U \ {1}. 



Proposition 6.1 ( i) For every (ft, e) e (0, 9) x [0, 1) and weU\jl|, there exists eo = toifi, e) > sufficiently 
small such that for all e € (0, €q] there holds 

i<o(7p-e,e) ~ L(yp,e) = V w (y/j, e ). 

(ii) For every e e [0, 1) and oj € U \ {1}, the total multiplicity of oj degeneracy of yp^ e (2n) for ft e [0, 9] 
is always precisely 2, i.e., 

2 y^Jp^Jt)) = 2, VweU\{l). 

y8e[0,9] 

Proof. Firstly, for (J3, e) e [0,9) x [0, 1), the operator A(J3,e) is a bounded perturbation of the operator 
-a r/dt 2 . Thus as -d 2 /dt 2 the operator A(J3, e) possesses only point spectrum, finite Morse index, each of its 
eigenvalues has finite multiplicity, and the only accumulation point of its spectrum is +oo. Consequently its 
eigenvalues are all isolated. 

(i) Fix weU\{l| and e e [0, 1). Let rj(fj) be a unit norm eigenvector belonging to an eigenvalue A(fj) of 
the operator A(fj, e) on D(oj, 2n) for /? near some /3q e (0, 9). Then as in Proposition 14.61 we obtain 



n 

{—A{p,e)rffl,rm> 
d/3 



/8=A> 



r (A(9,e)770S o ),^o)> 



2(9-A,) 3/2 

> 0. (6.3) 



Therefore eigenvalues of A(fj, e) on D{oj, 2n) for [3 € [0, 9) are strictly increasing in /?. 
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Note that by definition the Morse index k- - ^^(Aifio, e)) is the total multiplicity of the negative eigen- 
values of A(J3o, e), which is finite. 

Suppose ko = dim ker(A(/?o, e)) > holds on the domain D(co, 2k). By (16.31 ) there is a smallest positive 
eigenvalue A + (Bq) of the operator Aifio, e). Because A(B, e) depends analytically on B, we can choose e > to 
be small enough so that all the negative eigenvalues of the operator A(B, e) with B € \fi - 2e,B + 2e] c (0, 9) 
come only from perturbations of negative and zero eigenvalues of A(Bo,e), and are not perturbations from 
any eigenvalues of A(Bq, e) larger than or equal to A + (Bq). Therefore by (16.31 ) we obtain 



MA(Po ~ e, e)) - <p w (A(p , e)) = dimker(A08 o , e)), (6.4) 

together with Lemma [231 and (12.30b . which yields (i). 

(ii) Note first that by Lemma HjJ d23Ub . (ET8T ) and d3T9l . the operator A(B, e) on D(co, 2k) for B = or 9 
and co e U \ {1} is non-degenerate. 

Following our discussions in (i), at every Bq e (0,9) such that A(fio,e) is degenerate, the w-index must 
decrease strictly. But by (|3.7| i, (13.91 ) and Corollary 14.51 there exist at most two values B\ and B 2 at each of 
which the <x>-index decreases by 1 if B\ + B 2 , or the w-index decreases by 2 if B\ = B 2 . Therefore there 
exist at most two ySs in [0, 9] at which the operator A(B, e) degenerates by (i), which we denote by B\{e) and 
B 2 (e) e (0,9]. Thus by Corollary I4.5l again. we can choose e > small enough according to B\{e) and B 2 (e) 
in the above way so that we have (^j(A(0, e)) — (pco(A(B\(e) — €,e)), (f> ai {A{Bi{e), e)) — (p tJ (A(J32(fi) ~ £, e)), 
(f>tj(A(9, e)) = w (AO8 2 (e), e)), and (iMl ) holds for B replaced by Bi(e) and B 2 (e). Then this yields 



2 = w (A(O,e))-^(A(9,e)) 

= w (AOSi (e) - e, e)) - QJAifii (e), e)) + K{A{B 2 {e) - e, e)) - w (i(02(e), e)) 

= dim ker(A(jSi (e), e)) + dim ker(A(/3 2 (<?), e)) 

= v w (y j8l ( e)>e (27r)) + v w (y / j 2(e)ie (27r)) 

jSe[0,9] 

which proves the proposition. | 
Now set 

B(e,co)=A(9,e)~ L 2— — — S(t)A(9,e)^. (6.5) 

2(1 + ecos(f)) 

Be aware that B(e, co) depends on co, since so is A(9, e) on its domain D(co, 2k). Now we have 

Lemma 6.2 For any co boundary condition and e e CI, A(8,e) is co degenerate if and only if zL- is an 

79-" 



eigenvalue ofB(e, co). 

Proof. Suppose for e e Q, A(B, e)x = holds for some x e D(co, 2k). Let y = A(9, e)?x. Then by (14.111) 
we obtain 

A(9,e)k— r =+B(e,co))y(f) 

A(9, e) + — -S (t) x(t) 



^y/9~ r (3 2(l+ecosf) 

1. A(J3,e)x 

V9^S 

0. (6.6) 
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Conversely, if ( i + B(e, oj))y = 0, then x = A(9, e) ?y is an eigenfunction of A(fi, e) belonging to the 

V 9 ^ 



eigenvalue by our computations in (I6.6I ). | 

Theorem 6.3 For any oj e U, there exist two analytic oj degeneracy curves (fii{e, oj), e) in e e (-1, 1) with 
i = 1 and 2. Specially, each Bi(e,oj) is a real analytic function in e e (—1,1), and < Bj(e,oj) < 9 and 
ypi(e,w),eQ n ) iJ w degenerate for oj € U \ {1} and i = 1 or 2. 

Proof. For oj = 1, we have Bj(e, 1) = for i = 1 and 2, by Theorem II .H and (12- 31b . which is obviously 
analytic. 

Forw € U\ {1}, from dXT) we have w (A(O,e)) - c^(A(0,e)) = 2. On the other hand, c^(A(9,e)) = Oby 
(13 -9b - Recall that A{B,e) = "^=|> an d it is strictly increasing with respect to B by Lemma |4~4l This shows 

that, for fixed e e (-1, 1), there are exactly two values B = B\{e,oj) and B2(e,co) at which (16.6b is satisfied, 
and then A(yS, e) at these two B values is oj degenerate. Note that these two B values are possibly equal to 
each other at some e (compare with the figure in ifLTI ). which is not needed in this proof. 

Since /5 = 9 is a»-non-degenerate for any oj e U, we must have /?,(£, oj) + 9 for i = 1 and 2. By Lemma 
is an eigenvalue of B(e, oj). Note that B(e, oj) is a compact operator and self-adjoint when e 



-i 



is real. Moreover it depends analytically on e. By (§1 (Theorem 3.9 in p. 392), we know that , 1 : with 

i - 1 or 2 is real analytic in e. This in turn implies that both B\{e, oj) and faie, oj) are real analytic functions 
of e. ■ 

Now we can give 

Proof of Theorem 11.51 By Theorem 16.31 Bj(e, oj) is real analytic on c € [0, 1) for i - 1 or 2. That 
Bj(e, oj) — > as e —> 1 for i = 1,2 follows by the arguments in the proof of (ii) of Theorem ll.2l below in 
Section 8. And Corollary 14.5 I tells us that ico(.yp,e) is decreasing with respect to B e [0,9]. | 

Recall that the oj boundary condition is x(t) = tax(t + 2n), x(t) = ojxit + 2n). Let ifr(x)(t) = x(t + n), 
and obviously, ijj preserves the oj boundary condition. Also it is a unitary operator and = ip~ Y is given by 
ifj*{x){t) = x(t - n). One can show that 



it,*A(B,e)if,=A(B,-e). (6.7) 



In fact, 



rA(J3, e)^x(t) = ip* [~h ~h + ^77—^ ^( 3/ 2 + Q))) ^ + 

\ dt 1 2(1 + <?cos t) ) 

= if/* t-^I 2 -1 2 + — 1 — (3/ 2 + ^J9~^BS (t + n))) x(t + n) 

\ dt 1 2(1 - ecos(f + n)) J 

= A(B,-e)x(t). (6.8) 

By this property, we know that the oj degeneracy curve must be symmetric with respect to e = 0. 

When e = 0, the eigenvalues of ypfl(^2n) has been studied in §3.3. Specially A{B, 0) has no multiple 
eigenvalues for oj € U \ {±1} and < B < 9. So we have 

Theorem 6.4 For any fixed weU\j+l) and i = 1 or 2, the function Bj(e, oj) is real analytic and even on 
the interval (-1,1). 

It then follows that ^jS/( e > w )l<?=o = when weU\|+l|. But it is not the case when oj = -1, which we 
now turn to. 
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7 The - 1 degeneracy curves of elliptic Lagrangian solutions 

7.1 The two (o = -1 degeneracy curves 

For the a> = - 1 boundary condition, denote by g the following operator 

g(z)(t)=Nz(2n-t), (7.1) 

where N = ( Q ^ j. Obviously, g 2 = 1 and g is unitary on L 2 ([0,27r],R 2 ). Recall that E = D(-l,2n) is 
given by ( 12.1 11 ). One can check directly that 

A(fi,e)g=gA(fi,e). (7.2) 

Let E + = ker(g + I) and E- = ker(g - /). Following the studies in §2.2 and specially the proof of Theorem 
1.1 in El, the subspaces E+ and E- are A(J3, e)-orthogonal, and E = E + © Note that element z = (x, y) T 
in E- satisfies 

x{2n -t) = x{t), -y(2n -t) = y(t), V t e [0, 2tt]. 
Thus for all z = (x, y) T € we have 

x(n + t) = x(n - t), -y(n + t) = y(n - t), Vfe [0, 2n]. 

By the definition of z e D(-l,2/r), we have z(27r) = -z(0). Thus we have x(0) = x(2n) = -x(0) which 
implies ^(0) = and y(n) = 0. Similarly for all z = (x, y) T e E+ we have 

x{n + t) = -x{n - t), y(n + t) = y(n - t), V t e [0, 2n], 

and x(n) = and y(0) = 0. 

Therefore by the above discussions, the subspaces £L and E + are isomorphic to the following subspaces 
E\ and Ei respectively: 

E X = [z = (x,y) T e W 2 ' 2 ([0,^],R 2 ) | x(0) - 0, y(n) - 0}, (7.3) 
E 2 = [z = {x,y) T e W 2 ' 2 ([0, ;r], R 2 ) | x{n) = 0, y(0) - 0}. (7.4) 

For (J3, e) e [0, 9] x [0, 1), restricting A(J3, e) to E\ and E2 respectively, we then obtain 

<P-y{A(fi,e)) = cf>(A(/3,e)\ E t) + <f>(A(J],e)\ E2 ), (7.5) 
V-i(A(/3,e)) = v(A(/3,e)\ El ) + v(AQ3,e)\ E2 ), (7.6) 

where the left hand sides are the Morse index and nullity of the operator A(J3,e) on the space D(-l,27r), 
i.e., the -1 index and nullity of A(J3, e); on the right hand sides of (I7.5I )- (I7.6I ). we denote by <p{A{fl, e)\ Ej ) and 
v(A(J3, e)\ Ej ) the usual Morse index and nullity of the operator A(J3, e)\ Ej on the space Ej. 

By (14. 10b . we have 0_i(A(9,<?)) = and v_i(A(9,e)) = 0. Because all the terms in both sides of (1731 ) 
and (17.61) are the Morse indices and nullities which are nonnegative integers, we have 

0(A(9, e)\ El ) - 0(A(9, e)\ El ) = 0, v(A(9, ) = v(A(9, e)\ E2 ) = 0. (7.7) 

This shows that A(9, e)\ Ej with i - 1 or 2 is positive definite. 

Since the operator S(t) commutes also with the operator g, similarly we have 

S(t) - S (t)\ El ®S (t)\ El . (7.8) 
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So, for i - 1, 2, we obtain 

A(fi,e)\ E , = A(9,e)\ Ei + 



-S(t))\ Ej = V^s 



2(1 + ecost) 



A(9,e)\ Ei S(t)\ E . 
+ 



yj9^]3 2 (! + ecost) / 



(7.9) 



Now we want to compute the Morse index of A(0, e)\ Ej for i - 1, 2. By ( 13.71 ) and ( 17.51 ), we have 

cf>(A(0,e)\ El ) + <f,(A(0,e)\ E2 ) = 2, Vee(-l,l). (7.10) 
So the possible value of 0(A(O, e)\ Ej ) can only be 0, 1 and 2. By (13.81 ) and (17.61) . we obtain 

v(A(0,e)| £l ) = v(A(0,e)| £2 ) = 0, Vee(-l,l). (7.11) 
By the property of Morse index, we have 

<KM0,e)\ Ei ) = #4(0,0)1,,), Ve e (-1, 1), i = 1,2. (7.12) 
From (I3.43l l and (I7.6I ). we obtain 

v(A(3/4,0)| £l ) + v(A(3/4,0)| £2 ) = v_!(A(3/4,0)) = 2. (7.13) 
From the fact that t;E\ = E 2 and = £1 with £ denned as in (I6.7I ). we have 

y(A(3/4,0)| £l ) = v(A(3/4,0)| £2 ). 

Then we obtain 

v(A(3/4,0)| £i ) = l, for/ -1,2. (7.14) 
By ( 17.91 ), for any fixed e e (-1,1) and i - 1,2, ^_ ^fj£i j s increasing with respect to j3 as proved before. 

Y9-/3 

It has the same Morse index and nullity as those of Aifi, G)\ Er So we get 

MM* mi ^ J 1 ' if 0^^<3/4, r7 ,^ 

«A<ftQ)|*) = | 0j if ^> 3/4) ( 7 -!5) 

This shows that for -1 < e < 1 and / = 1, 2, by ( 17.121 ) we obtain 

0(A(O, e )| £ .) = l. (7.16) 

By the same idea as in the proof of Theorem 16.31 we get 

Proposition 7.1 The a> = -1 degeneracy curve (fii{e, - l),e) is precisely the degeneracy curve ofA(fi,e)\ Ej 
for i = 1 or 2. 

Following the results of lfl6l . lfT2l and lfl5l . we know that the curves (fii(e, -1), e) and (fi2(e, -1), e) 
intersect transversally at the point (3/4, 0). By symmetries of the w-index gap curves, we have 

Theorem 7.2 p { {e, -1) = p 2 (-e, -1) holds for allee(-l, 1). 

Remark 7.3 We can also compute (p(A(0, 0)\ El ) via the relation between Maslov-type index and Morse 
index. Let V\ = {(0, x,y, 0) | x,y € Rj and V2 = {{x, 0, 0,y) | x,y € R}. Then both of them are Lagrangian 
subspaces of the phase space R 4 with standard symplectic structure. From Theorem 1.2 of |@], we have 

<f>(A(0,0)\ El ) = MV 2 ,7o,o(OVU (7.17) 

where the right hand side is the Maslov index for paths of Lagrangian subspaces. Note that for {ft, e) = (0, 0), 
by (12.191 ). B(t) = B is a constant matrix. Then yo,o(0 - cxp(JBt) and its Maslov-type index can be computed 
explicitly as we did in §3.3. 
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7.2 -1 degeneracy curve bifurcations from (3/4, 0) as e leaves from 



By (13.221 ). -1 is a double eigenvalue of the matrix 73/4,0(2^). As studied by Roberts (cf. p.212 in lfT6l ) and 
Meyer-Schmidt (cf. Section 3 of 03]), there are two period doubling curves bifurcating out from (3/4,0) 
when e > is sufficiently small. In fTBTl . the tangent directions of these two curves are computed. Note that 
these two curves are precisely the - 1 index gap curves found by our Theorem ll.2l For reader's conveniences, 
here we give a simple proof on these two tangent directions based on our above studies. 



Proposition 7.4 The tangent directions of the two curves T s and T m bifurcating from (3/4, 0) when e > is 
small are given by 

V33 , V33 

P' s (e)\e=Q = T~ . /4(e)| e=0 = — T- 



Proof. To compute the slope of -1 degeneracy curve bifurcating out from (f3, e) = (3/4,0), let (J5{e),e) 
be one of the curve (say, the E\ degeneracy curve) with e e (—e, e) for some small e > 0, and x e e E\ be the 
corresponding eigenvector, that is 

AQ3(e),e)x e = 0. 



Here the space E\ is defined in (17.31 ) above. Thus there holds 

(A(J3(e),e)x e ,x e ) = 0. 
Then by direct computations, ker(A(3/4, 0)) n E\ is generated by xq - R(t)z(t) with 

7- V33 . 



z(t) - (- 



sin(?/2),cos(?/2)y . 



(7.18) 



(7.19) 



(7.20) 



Differentiating both sides of (17.191 ) with respect to e yields 

P(e)(—A(J3(e),e)x e ,x e ) + «— A(J3(e), e)x e , x e ) + 2(A(J3(e), e)x e , x' e ) = 0, 
op oe 

where fi'ie) and x' e denote the derivatives with respect to e. Then evaluating both sides at e = yields 

/3'(0)(—A(3/4,0)x ,xo) + (— A(3/4, 0)^o,^o> = 0. 
op oe 



Then by the definition ( 12.291 ) of Aifi, e) we have 



d 

d~e A(fi ' e) 



3 



= R{t)—K^ e {t) 

G8,e)=(3/4,0) °!P 

= R^—Kp^i) 

<J3,e)=(3/4,0) Oe 



Riff, 



O8,e)=(3/4,0) 



GM=(3/4,0) 



R(tf, 



(7.21) 

(7.22) 
(7.23) 



where R(t) is given in §2.1. By direct computations from the definition of Kp^{t) in (12.201 ). we obtain 

1 /-l 



C8,e)=(3/4,0) 



O8,e)=(3/4,0) 



2V33^0 1/' 
- cos 



t /6+ V33 \ 

6-V33J 



(7.24) 
(7.25) 
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Therefore from dT20T i and d7T22T >- d7T25T > we have 

3 d 
<— A(3/4,0)x ,*o> = (^K 3/4fi z,z) 
ofi ofi 



Jo 2V 



o 1 7 - V33 , , 
: cos 2 (f/2) — (( -— f sin 2 (?/2)]J? 



V33 2V33 



4V33 

and 

d d 
<— A(3/4,0)x ,x > - (— ^ 3 /4,oZ,z> 



d-(^) 2 ), (7-26) 



1 7 — a/33 1 

[-(6 + V33)( -— ) 2 cos(f) sin 2 (f/2) + -(6 - V33) cos(/) cos 2 (t/2)]dt 

Jo 4 4 4 

= "( 6 - V33-(6 + V33)(l^!) 2 ). 
16 

Therefore by (1X211 and (l7.26b - <T7TT7b we obtain 



,6 4 <7 " 27) 



V33 

yS'(O) = (7.28) 

By the above Theorem 17.21 the two -1 degenerate curves are symmetric with respect to the e = axis. 
Therefore the claim of the Proposition l7.4l follows from (I7.28I ). | 



8 Study on the non-hyperbolic regions 

Now we give proofs of the first halves of our main theorems ll.6l and [L2l 
Proof of (i) of Theorem 1 1.61 It follows from Theorems 16.41 and |7 . 2 1 | 

Proof of the first half of Theorem 11.21 Here we give proofs for items (i)-(iii) and (ix)-(x) of this 
theorem. 

(i) By Theorem 1 1.51 we have got the existence of two curves defined by (fii(e),e) and lim^i/ 3 ,-^) = 
with i = 1 and 2 such that yp^ e {2n) is degenerate with respect to co = -1 only on them. Note that here these 
two curves may coincide somewhere. Specially we define 

< fi,(e) = min{Pi(e),j82(e)} < fi m (e) = max^ie),^)} < 9, for e € [0, 1). (8.1) 

Thus (i) is proved. 

(ii) By the studies in §3.3, the only - 1 degenerate point in the (J5, e) segment [0, 9]x{0} is ifi, e) = (3/4, 0), 
which is a 2-fold - 1 degenerate point, and there hold 

Uijpfi) = 2, v_i(r A0 ) = 0, for fi e [0, 3/4), (8.2) 

i-i(r3/4,o) = 0, V-i(y 3 /4,o) = 2, (8.3) 

U(7p,o) - 0, v_i(r A0 ) = 0, for fi e (3/4, 9]. (8.4) 

Therefore 

fii(0) = 3/4, for i - 1 and 2. (8.5) 

By Meyer and Schmidt in |[T5l or our Proposition 17.41 the two -1 degeneracy curves bifurcating out 
from (fi, e) = (3/4, 0) when e > is sufficiently small must coincide with our curves T s and T m respectively. 
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Because these two curves bifurcate out from (3/4,0) in different angles with tangents - V33/4 and V33/4 
respectively when e > is small, they are different from each other near (3/4,0). By our Theorem 16.41 
these two curves r s and T m are real analytic with respect to e. Therefore they are different curves and their 
intersection points including the point (3/4, 0) can only be isolated. 

By Theorems 16. 3 l and [1.51 these two curves F s and Y m must tend to the segment [0, 9] x {1} from (3/4, 0) 
as e increases from and tends to 1. By the proof of our Theorem 11.81 in §8 below, for each e e [0, 1) the 
function /3t(e) denned by (11.51 ) satisfies < fi s (e) < fS m (e) < fiuie) < 9, and lim e ^i pu{e) = 0. Therefore the 
two curves T s and T m must tend to (0, 1) as e — » 1. 

(iii) By our studies on the segments {0} x [0, 1) and {9} x [0, 1) in §3 and the definitions of /^(e) and 
P m (e), the index i-i(yp >e ) must takes the claimed values 2, 1, and in (11.71) respectively when /? e [0,9] \ 
{fi s (e),fi m (e)} for each e e [0, 1). Note that when p = f3 s (e) or fl m {e), the -1 index claim (11.71 ) follows from 
(i) of Proposition 16.11 The last claim in (iii) follows from Proposition 16.11 

(ix)-(x) By the Bott-type formula (Theorem 9.2.1 in p. 199 of |QT|), we obtain 

= E ^ e) ' v k e N ' 

and 

where the last equality follows from Theorem 11.11 

Therefore by CCD, (ED and (ESJ), for e € [0, 1) we obtain 

(4, if <£<&(*>), 
<f> 2 = 3, if PAe)</3</3 m (e), (8.6) 
1 2, if Me)</3<9. 

Thus for (ft, e) € (0, 9] x [0, 1), the matrix yp^n) = e (2n) is non-degenerate with respect to the eigenvalue 
1, whenever (yS, e) T s U Y m . Therefore by (18.61 ) we can apply Theorem 12.5 K i.e.. Theorem 1.2 of Q) to get 
(ix) and (x). The rest parts of Theorem II .2 1 will be proved in the next section. | 

9 Study on the hyperbolic region 

In this section we study the hyperbolic region of yp, e {2n) in the rectangle [0, 9] x [0, 1). By the first halves 
of Theorems 11.21 and 11.6 I pro ved in the §8, the function denned by (11.51 ) satisfies 

Pm(e)<Me), Vee[0,l). (9.1) 

We have the following further results. 

Lemma 9.1 (i) I/O < J3i < fi% < 9 and yp l , e {2n) is hyperbolic, so does y^^ln). Consequently, the 
hyperbolic region ofyp^ e (2n) in [0,9] x [0, 1) is connected. 

( ii) For any fixed e e [0, 1), every matrix yp, e (2n) is hyperbolic iffikie) < < 9 for fiu{e) defined by di.5D . 
Thus ( li.8D holds and Tu is the boundary set of this hyperbolic region. 

( iii) We have 

v w (y^(2»r)) = 2, V^eU\{l}. (9.2) 
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Proof, (i) By Lemma 4.4, for any fixed co e U and e e [0, 1), the operator A(B,e) - i — A(J3, e) is 

Y9-jS 

self-adjoint on D(w, 27r) and increasing with respect to ft in the sense that 

A(J3 u e)<A(fi 2 ,e), if fiKfo. (9.3) 

Suppose yp ue (2n) is hyperbolic. This implies that A(J3\, e) is non-degenerate on D(a>, In) for every well. 
By (ix)-(x) of Theorem 1 1.21 it also implies fi m (e) < fi\. Thus by (12.30b . (13.71) . Corollary 14.51 and Theorem 
11.11 the w-index 4>aj(A(Pi,e)) = for all a> € U. Then A(fii,e) is positive definite on D(co,2k) for every 
a> e U. Therefore by (I9.31 l the operator A{^2,e) is positive definite too, and then is non-degenerate on 
D(a>, 2n) for all well. Therefore yp 2 ^ e {2n) must be hyperbolic and so does yp^ e (2n) for all /? € 9). 

Recall that along the segment {9} x [0, 1) the matrix y9 >e (27r) is hyperbolic by our Proposition II .41 There- 
fore the hyperbolic region of yp e (2n) is connected in [0, 9] x [0, 1). 

(ii) By the definition of fi\(e), there exists a sequence {/?,}, e N satisfying > fi\(e), — > f$k(e), and 
yp^e(2n) is hyperbolic. Therefore yp e {2n) is hyperbolic for every /? e (/3,t(e), 9] by (i). Then d 1 - 81 ) holds and 
is the envelop curve of this hyperbolic region. 

Now (iii) follows from (ii) and Proposition 16.11 and the proof is complete. | 
Corollary 9.2 For every e € [0, 1), we have 

£ v_ 1 ( 7Ae (2^)) = 2 W J] v_i(y Ae (2»r)) = 0. (9.4) 

Proof. Fix an e e [0, 1). If J3 s (e) < fi m (e), then we obtain 

^ v-i(yg,e( 27r )) ^ v_i(r/3 J ( e ), e (27r)) + v_i(7 j8m(e)>e (27r)) > 2. 
j8e(0A(e)] 

Thus (19.41) follows from (ii) of Proposition 16.11 

If p s (e) = fi m {e), then by (i) of Proposition 16. 1 1 we obtain V-i(ys m (e), e (27r)) = 2. Therefore we have 

2_j v -\-(7pA 2n )) ^ v-i(y j e m ( e ), e (27r)) = 2. 

/3e(0,/3„,(e)] 

Thus (19.41) follows also from (ii) of Proposition 16.11 | 
Now we can give the 

Proof of the second half of Theorem 11.21 Here we give the proofs for the items (iv)-(viii) and (xi) of 
this theorem. 

Note that Claims (iv) and (v) of the theorem follow from (19.11 ) and Lemma |9T| 

(vi) In fact, if the function fik(e) is not continuous in e e [0, 1), then there exist some e € [0, 1), a 
sequence {e t \ i e N} c [0, 1) \ {e) and j3 e [0, 9] such that 

Pkied -> Po ±Pk{e) and e t -> e as i -» +oo. (9.5) 

We continue in two cases according to the sign of the difference Po -/?*(£). 

By the definition of /?jt(e;) we have cr^^)^. (2ff))nU ^ for eveiy e,-. By the continuity of eigenvalues 
of yp k {e,),ei(2n) in / and ( 19.51 ), we obtain 

(r^gO)) nU#0. 
Then by Lemma |9~T1 this would yield a contradiction if /?o > 
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Now suppose /?o < jSfe(e). By Lemma I9TT1 for all i > 1 we have 



a-{yp, ei {2n)) n U = 0, V p e 9]. (9.6) 

Then by the continuity of p m (e) in e, ( 19.61 ) and the definition of /?o, we obtain 

Pmie) <Po< /3 k (e)- 

Let ojo e o~(yp k (eie(2jr)) n U, which exists by the definition of 

Let L = {(fi,e) \ p e (p k (e),9]}, V = {(9,e) | e e [0, 1)}, and U = {(fi,ed | yS € (&(«/), 9]} for all i > 1. 
Note that by d3T9l , ( 14.101 ), Corollary @3] Proposition I6TT1 Lemma|9Tll and the definitions of p k (e{) and p k {e), 
we obtain 

^ (7/?, e ) - y m (7/},e) = 0, V OS, e) 6 L U V U (J L,. (9.7) 

!>1 

Specially we have 

^ (7A(g),g) = and v^Cr^xg) > 1. 

Therefore by Proposition 16. 1 1 and the definition of coo, there exists p € (Po>Pk(e)) sufficiently close to p k (e) 
such that 

iwo(T^) = ^ (7A(g),g) + i / W0 (r / j,(e),e(27r)) > 1. (9.8) 

This estimate ( 19.8b in fact holds for all ft e []3,p k (e)) to °- Note that (/?, £) is an accumulation point of U, >\Lj. 
Consequently for each i > 1 there exists (/?,-, a) € L, such that yp uei £ ^*2^(4) is coo non-degenerate, fit — > /? 
in R, and y^.^, — > y^ - in ^2^(4) as / — > oo. Therefore by (19.71 ), (19.81 ), the Definition 5.4.2 of the ^o-index of 
a>o -degenerate path y^ ~ on p. 129 and Theorem 6. 1 .8 on p. 142 of ifTTTl , we obtain the following contradiction 

1 < L (yfe) £ i(oo<ypt#) = °' 

for i > 1 large enough. Thus the continuity of p k (e) in e e [0, 1) is proved. 

(vii) To prove the claim lim^ i p k (e) = 0, we argue by contradiction, and suppose that there exist e,- — > 1 
as i — > +oo,/3 > 0, such that lim^oo Pki^i) = Po- Then at least one of the following two cases must occur: 

(A) There exists a subsequence of{e\\ such that p k {ej + \) < p k (e()for all i e N; 

(B) There exists a subsequence {£,-} of{ej] such that p k (e~i) ^ Pki^i+i) f° r all i e N. 

If Case (A) happens, for this po by Theorem 1 1 . 7 1 there exists eo > sufficiently close to 1 such that 
yp ^ e {2n) is hyperbolic for all e e [eo, 1). Then yp, e (2n) is hyperbolic for all (J3, e) in the region [Po, 9] x [eo, 1) 
by Lemma |9~T1 But by the monotonicity of Case (A) we obtain 

Po < Pk(e i+m ) < Pk(ei) VmeN, 

Therefore (p k (ei +m ), e,- +m ) will get into this region for sufficiently large m > 1, which contracts to the defini- 
tion of ySyt(e, +m ) in (1 1 -5b - 

If Case (B) happens, fix an subindex i, by Theorem ll.7l and the same argument as in Case (A) there exists 
an eo > sufficiently close to 1 such that yp^ e (2n) is hyperbolic for all (fi, e) in the region [p k {e\), 9] x [eo, 1). 
Then by the monotonicity of Case (B) we obtain 

Pt&i) < J3 k (e i+m ) <j3 VmeN. 

Therefore (p k (ei +m ), e (+m ) will get into this region for sufficiently large m > 1, which contracts to the defini- 
tion of p k {e i+m ) in d 1 -5b - Thus (vii) holds. 

(viii) By our study in §3.3, we have (1,0) e \ T m . Thus there exists an e e (0, 1] such that p k (e) > 
p m (e) for all e € [0, e). Therefore T k is different from T m when e e [0, e). 
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(xi) Let eo e [0, 1) and /3 m (eo) < < Pki^a)- Then M = y^ eo (2n) is not hyperbolic by Lemma |9T| and 
thus at least one pair of its eigenvalues is on U. Note also that no eigenvalues of M can be ± 1 by Theorem 
Oand Corollary E2 Write 

cr{M) = {A l (p Q ),A l (fi T\A 2 (fi Q ),A 2 (p Q T 1 }. (9.9) 

Thus we can assume A\ = A\(J5q) e U \ R and the other pair of eigenvalues satisfy A 2 = A 2 (fio) e (U U R) \ 
{±1,0}. 

Claim. A 2 (J3 Q ) e U \ R. 

In fact, if not, we assume A 2 (j3o) e R \ {±1, 0}. 

In this case, M has normal form R(ff)oD(A 2 ) e Q°(M) for some G e (0, n) U (n, 2n). Thus by d376l . 
Theorem [TTTJ and (iii) of our Theorem II .21 we obtain the following contradiction: 

= i-i(7/W 

= h(yp ,eo) + s + M (i) - s-^e^ 10 ) + s^ ±vrTe ) - s M (-i) 

= + 0-S- m (e^ d ) + S + m (e^ s )-0 
= ±1, 

where the last equality follows from Lemma 9. 1.6 on p. 192 and (5) of List 9.1.12 on p. 198 of ifTTI . Therefore 
the claim is proved. 

Now from A\(fi ) and A 2 Q3o) e U \ R, the matrix M has basic normal form R(9i)oR(9 2 ) e Q°(M) for 
some 6\ and 9 2 e (0, n) U (n, 2n). Then by the study in Section 9. 1 of (TTJ, we obtain 

= i-\(yfs , eo ) 

= h(yp ,eo) + S* M (l) - 5 R(ei) (e ±vrTe ') +5^ ( e ±VZTei ) 

By Lemma 9. 1.6 on p. 192 and <5> of List 9. 1. 12 on p. 198 of HTJ again, the right hand side of d9~T0l ) would 
be ±2, if both 6\ and G 2 are located in only one interval of (0, n) and (n, 2n). Thus we must have 8\ € (0, n) 
and 9 2 e (tt, 2tx). Let w = exp( V— 
If 2n - 9 2 = 6i, we then obtain 

^ v w (r^, eo ) ^ 2j v ^y^o) + v ^(r/?o,^o) > i + 2. 

0</S<yS 0<P<p m {e ) 

It contradicts to Lemma I97T1 and proves 2n - Q 2 + Q\. 

By the study in Section 9. 1 of ifTTll again, if 2/r - 9 2 > 6\, for oj - exp( V-I#i) we obtain 

< L(y^ eo ) = /i(y/W + ^m(D - S~ m) (e^) = -S^^ 01 ) = -l. 

This contradiction proves that the only possible case is 2/r - 2 < 6\ . 
The proof of Theorem [L2]is complete. | 
Now we give 

The proof of (ii) of Theorem ll.6[ By (v) of Theorem II .21 the curve I* for e € [0, 1) is the boundary 
curve of the hyperbolic region of yp^ e {2n) in the (/?, e) rectangle [0,9] x [0, 1). By the definition (I1.5I ). the 
curve T)t is also the envelop curve of the oj degeneracy curves for all oj e U \ { 1 } from right hand side 
of the rectangle [0,9] x [0, 1). Then (i) of Theorem 11.61 implies that T* can be continuously extended into 
[0, 9] x (-1, 0] so that it is symmetric with respect to [0, 9] x {0}. | 

The next lemma is useful in the proof of Theorem [ 
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Lemma 9.3 If yn 0>e (27r) ~ M 2 (— l,c) holds or it possesses the basic normal form N\{—\,a)oN\{—\,b) for 
some (fio, e) € (0, 9) x [0, 1) and a, b € R and c € R 2 , then yp^iln) is hyperbolic for all fl € (/5o, 9]. 

Proof. Note that the basic normal form of M 2 (-l,c) is either Ni(-\,a)oNi(-l,b) or Ni(-l,a)oD(A) 
for some a, b e R and > A ± — 1. Thus for any w e U\ by Corollary 14.51 (I2.311 i. and the study in 
Section 9.1 of ifTTl . we obtain 

< ico(7{l ,e) = h(7fio,e) + ^mC 1 ) ~ S~ M {0)) = S M (oj) < 0, 

where M = yp^ e {2n). This proves i w (yp , e ) - for all oj e U. Note that ^(Affio, <?)) - ia)(yp , e ) an d 
VuiAiPo, e)) = v'dypoA 2 ^ follow from (I230I) . ffia and (|4~T3T> . 

Now from w (A(j8 o , e)) = and (ii) of Lemma [4~4l we obtain A(/3, e) > for all /? e (jS , 9] on D(oj, 2n) 
with w e U. Therefore v w (y /3 e (2^)) = v w {A(p,e)) = holds for all yS e (J3q,9] and <y e U, and thus the 
lemma follows. ■ 

Now we can give 

Proof of Theorem |L8l (i) Let e e [0, 1) satisfy J3 s (e) < /3 m (e). Then Corollary ^implies V-i(y Me)>e ) = 

I. As the limiting case of Cases (ix) and (x) of Theorem 1 1.21 the matrix M = yp s ( e ),eQn) must have all eigen- 
values in U, and possesses its normal form either M « Mj{-l,c) for some c 2 + 0, or M « A^i(-1, l)o/?(0) 
for some e (7r, 27r), where to get the second case we have used the Figure 2.1.2 on p. 50 of ifTTIl and the fact 
iVi(-l, 1) e Sp(2)° 1 _ in that figure. 

Note that M k M2(-1,c) for some c 2 + can not hold by Lemma |931 and the fact f3 s (e) < p m (e). 
The following is a direct proof on this fact. In this case, its basic normal form is Ni(-l,a)oD(A) for some 
a e {-1, 1} and > A + -1. Therefore by Theorem [lj] (iii) of Theorem [L2] and (3> and <4> of List 9.1.12 
on p. 198 of IfTTl . we obtain the following contradiction 

1 = i-i(7js,M*) = hiypM,e) + s mW ~ s Nl (-i,a)(-V = -^(-i,^- 1 ) ^ °- 

Thus M«iVi(-l, 1)<>R(9) must hold for some 9 € (n, 2n), so M is spectrally stable and linearly unstable. 

(ii) Let e € [0, 1) satisfy f3 s (e) = fl m {e) < fik(e)- As the limiting case of the Cases (ix) and (xi) of Theorem 

I I. 21 and Corollary 19.21 the matrix M = yp s { e ), e Qri) must have basic normal form either N\(—\,a)oN\(—\,b) 
for some a and be {-1,1}, or -l2<>R(ff) for some 9 e (n, 2n), where we have used the Figure 2.1.2 on p. 50 
of IfTTl . But the first case is impossible by Lemma 1931 Therefore M ~ -l2<>R(9) holds for some 9 e (n, 2n), 
and it is linear stable and not strongly linear stable. 

(iii) Let e e [0, 1) satisfy fi s (e) < fi m {e) < fik(e)- As the limiting case of Cases (x) and (xi) of Theorem 
11.21 the matrix M = yp m (e\eQ n ) must satisfy either M * N\(-\,-\)oR{6) for some 9 e (n,2n), or M ~ 
M 2 (-l,c) with c 2 + 0, where we have used the Figure 2.1.2 on p.50 of IfTTl and the fact iVi(-l,-l) € 
Sp(2)° 1 + in that figure. Here the second case is also impossible by Lemma [931 and the conclusion of (iii) 
follows. 

(iv) Let e e [0, 1) satisfy fi s (e) < fi m (e) < fik(e)- As the limiting case of the Cases (v) and (xi) of 
Theorem 11.21 the matrix M = yp k { e ), e Qjf) must have Krein collision eigenvalues cr(M) = {A\, A\, A2, A2} 
with/lj = I 2 = e^for some 9 e (0, n) U {n,2n). Here we have used Theorem II. H and Corollary 19.2 | to 
exclude the possibility of eigenvalues ± 1 . Therefore for this angle 9, the matrix M must have its normal 

form N2(co,b) for oj - e^ e and some 2x2 matrix b - ( , 1 , 2 ), which is of the form (25)-(27) by 

V b-i b 4 J 

Theorem 1.6.11 on p.34 of |HJ. Because (7 2 o(-/ 2 ))~ 1 A r 2 (e VrTe , b)(I 2 o(-I 2 )) = N 2 (e^ 27r -^, b) holds for 
/ b\ -b2 \ 

~ \ b b /' We ° an a ^ wa y s suppose 9 e (0, n) without changing the fact M « A^ 2 (o), b). 
Note that by (13.71 ). (13.91 ). Corollary 14.5 l and Proposition 16.11 we have iw(yp k (e),e) - 0. 
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Now if Z?2 - b 3 - 0, by Lemma 1.9.2 on p.43 of (Til, we get v Ll) (N2(u,b)) - 2, and then N%{o),b) has 
basic normal form R(6)oR(2n - 0) by the study in Case 4 on p.40 of ifTTTl . Thus we arrive at the following 
contradiction 

= L(7Me),e) = h(yMe),e)+SMW-SR(0)(u)-SR(e)(u) < -1, 

by Lemma 9. 1.6 on p. 192 and <5> of List 9. 1. 12 on p. 198 of UTTl 
Therefore - b$ ± must hold. Then we obtain 

= L(7/3 k (e),e) = il(yf} k (e),e) + Sm(1) - S N2{tt)h) (a>) = S N2(ajh) (oj). 

By (6) and (7) in List 9. 1. 12 on p. 199 of IfTTTl . we obtain that Ni(cd, b) must be trivial as in our discussion in 
§2.1. Then by Theorem 1 of BUI , the matrix M is spectrally stable and is linearly unstable as claimed. 

(v) Let e e [0,1) satisfy p s {e) < fi m (e) = fikie)- Note first that -1 must be an eigenvalue of M = 
y/3 k (e),e(2n) with geometric multiplicity 1 by Corollary 19.21 As the limiting case of Cases (v) and (x) of 
Theorem 11.21 the matrix M must satisfy either M « M2(-l,b) with b\,b2 € R and bj + 0, and thus is 
spectrally stable and linearly unstable; or M » iVi(-l, a)oD(A) for some a 6 {-1, 1} and -1 t A < 0. 

Then in the later case we obtain 

- i-i(yp k (e),e) = h(7M^) + - S Ni( _ l a) (-l) = -5^ l( _ 1>a) (-l). 

Then by (3) and (4) in List 9.1.12 on p. 199 of IfTTTl . we must have a - 1. This case can be seen in Figure 
2.1.2 on p.50 of IUJ with the fact N\(-l, 1) € Sp(2)° 1 . Thus M is elliptic-hyperbolic (EH) and linearly 
unstable. 

Note that by the above argument, the matrix M2{-\,b) has also basic normal form N\{-\, \)oD(A) for 
some -1 + A < 0. 

(vi) Let e e [0, 1) satisfy fi s (e) = fi m (e) = fik(e). As the limiting case of Cases (v) and (ix) of 
Theorem 11.21 -1 must be the only eigenvalue of M - yp k ( e ),eQ-n) with v_i(M) = 2 by Corollary 19.21 
Thus the matrix M must satisfy M ~ M2{-\,c) with C2 - and v_i(M2(-l,c)) = 2 by Subsection 
2.1; or M * N\(-\,a)oN\(-\,b) for some a and b e {— 1, 1}. In both cases, M has basic normal form 
N\{-\,a)oN\{-\,b) for some a and b e {-1, 1}. Thus we obtain 

= i-i{yp k (e),e) 

= h(yfi k {e),e) + SmQ.) - S Nl( _ Ua) (-l) - S^ i( _ i b) (-l) 

Then by (3) and (4) in List 9.1.12 on p. 199 of ifTTl . we must have a = b = 1 similar to our above study for 
(v). Therefore it is spectrally stable and linearly unstable as claimed. 

The proof is complete. | 

We give finally the 

Proof of Theorem [L9j Note first that y^ e is analytic in both ft and e. Since the property of the spectrum 
for a 4 x 4 symplectic matrix being complex saddle is an open condition in Sp(4), the set I e in the theorem 
must be open in for any fixed e e [0, 1). Thus for any fixed e e [0, 1), it suffices to show I e + 0. We argue 
by contradiction and suppose I e = 0. Then for any (fi, e) e (0, 9] x [0, 1), all the eigenvalues of yp >e form two 
pairs and are located on the union (R\ {0}) UU. By our Theorem ll.il the matrix y^ e is non-degenerate when 
P > 0, and then it has no eigenvalue 1 at all. But by our Proposition 1 1.41 the matrix y^^ e {2n) has a pair of 
double positive eigenvalues not equal to 1 for all e e [0, 1). Therefore fix an e e [0, 1). By the continuity of 
the spectrum in /3, the matrix y y g, e (2^') would have only positive eigenvalues not equal to 1 for all (3 € (0, 9]. 
This then contradicts to our Theorem 11.21 which yields the existence of the eigenvalue - 1 of yp t( ,(2n) for 
some /? € (0, 9), and completes the proof. | 
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10 More observations 



Here we describe briefly some more results of [141 of Martinez, Sama and Simo on the Lagrangian triangular 
homographic solutions in the Newton potential case. 

For e < 1 and close to 1, the system is HH for any /3 except in a neighborhood of some critical value 
which, numerically, appears to be equal to 6. Some interesting tangencies are also observed near the corre- 
sponding boundaries. 

(i) The tangency at (J3, e) = (0, 1) between the e- vertical axis and the curve which separates the EE and 
EH domains is of the form e - 1 - C/?S ; 

(ii) The tangency at (J3, e) = (0, 1) between the e = 1 horizontal line and the curve which separates the EH 
and HH domains is of the form e - 1 - C/3 4 ; 

(iii) For fixed /3 € (0, 9), the matrix yp^(2n) is HH if 1 - e > is small enough under a special "non- 
degenerate" condition, which is defined in their Lemma 5 on p. 663 of |[T4l . i.e., d g ± and e g ± 
there. It seems to us that it is not easy to verify this non-degenerate condition, and that the point 
(yS, e) - (6, 1) is a possible degenerate point is only checked numerically in |[T4l . 

(iv) The tangency at ifi, e) = (9, 0) between the /5 = 9 vertical line and the curve which separates the HH 
and CS domains is of the form e = C(9 - (3)*. 

In all the above expressions C denotes suitable constants. Furthermore there is a point of contact of four 
different types of domains located approximately at (1.2091,0.3145). 

In the current paper, we have proved the non-degeneracy of the elliptic Lagrangian triangular solutions. 
We have also proved the global existences of separation curves T s in (i), T m and in (ii). Our Theorem 1 1.7 1 
is related to (iii). 

For a) e U\{1}, we showed that there are two nontrivial degeneracy curves (possibly tangent to each other 
at isolated points) f3\{e,oS) and P2{e,aS) (a) - -1 in Theorem 1 1.2 1 and general a> in Theorem 11.51 ) which are 
real analytic in e € [0, 1). These o» degeneracy curves actually yield a foliation of the non-hyperbolic region 
of y^ e (2n) in the (/?, e) rectangle [0,9] x (-1, 1), when a> runs through U. We have conducted numerical 
computations to see this interesting phenomenon according to our above analysis for (ft, e) € [0, 9] x (-1, 1). 
Specially in the Figure 4, we pick up certain figures from such computations for readers. It is interesting to 
know how these degeneracy curves behave under the variation of co, which we leave for future studies. 

In summary, many problems observed numerically already deserve pursuit further. For example, more 
precise properties of degeneracy curves including their asymptotic behaviors, possible intersections and 
variations with respect to co, including the above mentioned interesting properties. In this paper, we have 
not considered separations between HH and CS either. We may study these problems in some forthcoming 
papers, and we believe that the ideas and the method we have developed here can also be used to linear 
stability problems for other solutions of the ?i-body problems and systems with periodic coefficients. 
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